Lane line marking data fusion splicing method, medium, device and program product

Abstract

The invention provides a lane line marking data fusion and splicing method, a medium, equipment and a program product, and belongs to the technical field of intelligent traffic, the method comprises the following steps: obtaining road image data information shot by a multi-view vehicle-mounted camera in real time, and marking a road lane line; converting the point coordinate of each lane line to a laser radar coordinate system; sequentially calculating the intersection point of the line or the extension line of the line on the X axis and the forward intersection angle of the line or the extension line of the line and the X axis for each lane line; a sliding window algorithm is adopted to carry out lane matching on double yellow lines, and then lane matching is carried out on other single lane lines; combining the close line segments of the two lines in the matching combination through a close point fusion algorithm; and combining the two line segments in each matching combination into one line segment. According to the method, the matching precision and efficiency of the lane lines among the multiple projections of the radar system can be effectively improved, the precision and robustness of the multi-view lane line fusion technology are improved, and the burden of technicians is greatly reduced.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a method, medium, equipment, and program product for fusion and splicing lane line marking data. Background Technology

[0002] Lane detection aims to identify the location information of lane lines in road scene images and plays a crucial role in the perception module of autonomous driving systems. As researchers delve deeper into the exploration and research of higher-level autonomous driving systems, autonomous driving technology is advancing towards a higher level. Against this backdrop, end-to-end autonomous driving methods are rapidly developing and gradually becoming a new research hotspot. Correspondingly, bird's-eye view (BEV) based lane detection methods are gradually replacing 2D lane detection methods, becoming the current research focus and cutting-edge direction.

[0003] With increasingly stringent requirements for task accuracy, detection algorithms are placing more stringent demands on the quantity and quality of data needed for the training phase. One method for rapidly acquiring BEV (Battery Electric Vehicle) labeled data is to use existing, effective 2D lane detection methods to extract 2D lane lines from images from different viewpoints. Then, combining the camera's intrinsic and extrinsic parameters, these 2D lanes are projected onto the BEV's viewpoint using inverse perspective mapping (IPM) and provided to the end-to-end model. However, a problem arises: the same lane line in the physical world can appear in images from different viewpoints. When the corresponding labeled data is projected onto the BEV after transformation, multiple lines are generated. Furthermore, due to camera calibration errors, these results cannot perfectly overlap. This severely impacts the training results and accuracy of end-to-end autonomous driving methods. Therefore, effectively stitching together 2D labeled results to generate BEV lane line labeled data, thereby ensuring data quality while efficiently reducing labeling costs, has become a crucial challenge that urgently needs to be addressed in the development of lane detection technology for higher-level autonomous driving systems.

[0004] Existing technology CN114299300A discloses a lane line detection method, apparatus, computer device, and storage medium. The method includes: extracting lane line features from image data, where the image data is image data collected by multiple acquisition devices, including lane line data; fusing the lane line features to obtain lane line fusion features from a bird's-eye view; determining lane line classification results and lane line direction classification results based on the pixel level of the lane lines in the lane line fusion features; and determining lane line instances based on the lane line classification results. This method fuses lane line features into lane line fusion features from a bird's-eye view using a pre-trained transformer network model, but it suffers from problems such as large model memory consumption, high equipment requirements, and long processing time.

[0005] Existing technology CN117237889A discloses a method, system, and storage medium for optimizing lane line annotation based on autonomous driving. It utilizes real-time road image data acquired by an onboard camera and real-time road point cloud data acquired by an onboard LiDAR. A thresholding algorithm is used for lane line annotation, resulting in lane line image annotation data. An improved Octree algorithm is then used for lane line annotation, resulting in lane line point cloud annotation data. The lane line point cloud annotation data is projected onto the lane line image annotation data for fusion, yielding fused lane line annotation information. However, this method, which fuses image and point cloud annotation data, cannot solve the problem of lane line fusion from different viewpoints. Summary of the Invention

[0006] This invention aims to solve at least one of the technical problems existing in the prior art, and proposes a method, medium, equipment and program product for fusion and splicing lane line marking data.

[0007] In a first aspect, embodiments of the present invention provide a method for fusing and stitching lane line marking data, comprising the following steps:

[0008] S100: Real-time acquisition of road image data information captured by multi-view vehicle-mounted cameras when vehicles are driving on the road, and lane line annotation of road image data from all perspectives to obtain lane line image annotation data information.

[0009] S200: Read the lane line image annotation data and convert the point coordinates of each lane line to the lidar coordinate system;

[0010] S300. In the lidar coordinate system, calculate the intersection point of the line or the extension of the line on the X-axis for each lane line, as well as the angle at which the line or the extension of the line intersects the X-axis in the positive direction.

[0011] S400. Based on the intersection point and angle, use the sliding window algorithm to perform lane matching for the double yellow lines, and then perform lane matching for the remaining single lane lines.

[0012] S500: Merge adjacent line segments of two lines in a matching combination using the nearest point fusion algorithm;

[0013] S600: Traverse each matching combination and merge the two line segments in each matching combination into one line segment.

[0014] Furthermore, in step S100, road image data information is acquired in real time based on the forward-looking and panoramic six-lens vehicle-mounted cameras.

[0015] Furthermore, in step S100, the labelme tool is used to annotate road lane lines to obtain lane line image annotation data information in labelme format.

[0016] Further, in step S300, the intersection points of each lane line or its extension on the X-axis are calculated sequentially according to the X-axis direction, as well as the angles at which the lines intersect the positive direction of the X-axis, to obtain a list of intersection point coordinates and a list of angles.

[0017] Furthermore, in step S400, the lane matching of double yellow lines using the sliding window algorithm includes: taking two lists, the intersection coordinate list and the angle list, as input, using the sliding window algorithm to find pairs of elements from the input data whose intersection coordinates and angles are close, and matching the corresponding lane lines into pairs of double yellow lines.

[0018] Furthermore, the window is set to contain multiple consecutive data points. The window is slid to the right sequentially, and the difference between the maximum and minimum values ​​in the intersection coordinate window and the angle window is calculated respectively. If the difference between the intersection coordinates and the angle are less than the preset thresholds, then it is determined that the four lines contain a pair of double yellow lines. Then, the pairing information of the double yellow lines is further determined from the window through the following operations: find the minimum value in the intersection coordinate window, then find the element with the closest difference, put the index of these two elements in the original intersection coordinate list as a key-value pair into the matching dictionary, and then put the indexes corresponding to the remaining two elements into the matching dictionary.

[0019] Furthermore, in step S400, lane matching for the remaining single lane lines includes: traversing all lane lines except for the double yellow lines, finding the curve with the closest intersection coordinates (excluding itself), and saving the matching information; if two lines are the closest to each other, and the difference between their corresponding intersection coordinates and the difference in angle are less than preset thresholds, then they are paired and added to the matching dictionary.

[0020] Further, step S500 includes: traversing each matching combination in the matching dictionary; for two lines in the combination, calculating the distance between each point on one line and each point on the other line; finding the nearest point on the other line for each point on one line; if the distance to the nearest point is less than a set threshold, replacing the coordinates of that point on one line with the coordinates of the nearest point on the other line.

[0021] Further, in step S600, merging two line segments into one line segment includes: calculating the lengths of the two line segments in the y-axis direction, with the longer line segment being the long line and the shorter line segment being the short line; firstly, removing points located between adjacent points of the long line from the short line to obtain a first processed line segment; then removing points located between adjacent points of the first processed line segment from the long line to obtain a second processed line segment; finally, merging the first processed line segment and the second processed line segment, and sorting them according to the y-coordinate to generate a merged line segment.

[0022] In a second aspect, embodiments of the present invention provide an electronic device, comprising:

[0023] One or more processors;

[0024] Memory, used to store one or more programs;

[0025] When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described above.

[0026] Thirdly, embodiments of the present invention provide a computer-readable medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps in the method described above.

[0027] Fourthly, embodiments of the present invention provide a computer program product including computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the method described above.

[0028] The lane line annotation data fusion and stitching method, medium, equipment, and program products provided by this invention acquire road image data information captured by multi-view vehicle-mounted cameras in real time while vehicles are driving on the road, and annotate the road image data from all perspectives to obtain lane line image annotation data information; read the lane line image annotation data, and transform the point coordinates of each lane line to the lidar coordinate system; in the lidar coordinate system, calculate the intersection point of the line or its extension on the X-axis for each lane line, as well as the angle of the positive intersection of the line or its extension with the X-axis; based on the intersection point and angle, use a sliding window algorithm to create lanes for double yellow lines. The process involves matching lanes, then matching lanes for the remaining single lanes; merging adjacent segments of two lines in a matching combination using a nearest neighbor fusion algorithm; iterating through each matching combination and merging the two segments in each combination into one; and finally, converting the labeled data to the radar coordinate system and then fusing the lane lines in the radar coordinate system. This effectively improves the matching accuracy and efficiency of lane lines across multiple projections in the radar system, enhancing the accuracy and robustness of multi-view lane line fusion technology. The application of this method significantly simplifies the data labeling process, greatly reducing the burden on technical personnel and freeing them from this highly repetitive and labor-intensive task. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating a method for fusing and stitching lane line marking data according to an embodiment of the present invention.

[0030] Figure 2 An example diagram illustrating the multi-view lane line annotation data stitching method provided in an embodiment of the present invention;

[0031] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0033] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0034] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0035] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0036] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0037] This invention provides a method for fusing and stitching lane line marking data. Figure 1 This is a flowchart illustrating a lane line marking data fusion and stitching method provided in an embodiment of the present invention. The method includes the following steps:

[0038] S100: Real-time acquisition of road image data information captured by multi-view vehicle-mounted cameras when vehicles are driving on the road, and lane line annotation of road image data from all perspectives to obtain lane line image annotation data information.

[0039] In one embodiment, road image data information is acquired in real time based on a forward-looking and panoramic six-lens vehicle-mounted camera.

[0040] In one embodiment, the labelme tool is used to annotate road lane lines, resulting in lane line image annotation data information in labelme format.

[0041] S200: Read the lane line image annotation data and convert the point coordinates of each lane line to the lidar coordinate system.

[0042] S300. In the lidar coordinate system, calculate the intersection point of the line or its extension on the X-axis for each lane line, as well as the angle at which the line or its extension intersects the X-axis in the positive direction.

[0043] Specifically, following the x-axis direction, calculate the intersection points of each lane line (l1, l2, l3, ..., ln, ..., lN) or its extension on the x-axis, as well as the angles at which they intersect the positive x-axis. This yields a list of intersection point coordinates (x1, x2, x3, ..., xn, ..., xN) and a list of angles (r1, r2, r3, ..., rn, ..., rN). ln represents the nth lane line, xn represents the intersection point of the nth lane line or its extension on the x-axis, and rn represents the angle at which the nth lane line or its extension intersects the positive x-axis.

[0044] S400. Based on the intersection point and angle, a sliding window algorithm is used to perform lane matching for the double yellow lines, and then lane matching is performed for the remaining single lane lines.

[0045] In one embodiment, see Figure 2 As shown in Operation 1, lane matching for double yellow lines using the sliding window algorithm includes: taking two lists, an intersection point coordinate list and an angle list, as input, using the sliding window algorithm to find pairs of elements from the input data whose intersection point coordinates and angles are similar, and matching the corresponding lane lines into pairs of double yellow lines.

[0046] Specifically, the window is set to contain multiple consecutive data points (e.g., the window contains 4 consecutive data points). The window is then slid to the right sequentially (e.g., the intersection coordinate window is "x1,x2,x3,x4" in the first processing and "x2,x3,x4,x5" in the second processing). The difference between the maximum and minimum values ​​in the intersection coordinate window and the angle window is calculated respectively. If the difference between the intersection coordinates and the angle difference are both less than a preset threshold, then it is determined that these four lines contain a pair of double yellow lines. Then, the pairing information of the double yellow lines is further determined from the window (e.g., x1,x2,x3,x4) through the following operations: find the minimum value in the intersection coordinate window, then find the element with the closest difference, put the indices of these two elements in the original intersection coordinate list as key-value pairs into the matching dictionary, and then put the indices corresponding to the remaining two elements into the matching dictionary.

[0047] In one embodiment, lane matching for the remaining single lane lines includes: traversing all lane lines except for the double yellow lines, finding the curve with the closest intersection coordinates (excluding itself), and saving the matching information; if two lines are the closest to each other, and the difference between their corresponding intersection coordinates and the difference in angle are less than preset thresholds, then they are paired and added to the matching dictionary.

[0048] S500: Merges adjacent line segments of two lines in a matching combination using a nearest-neighbor fusion algorithm.

[0049] In one embodiment, see Figure 2Operation 2 includes: traversing each matching combination in the matching dictionary; for two lines in the combination, calculating the distance between each point on one line and each point on the other line; finding the nearest point on the other line for each point on one line; if the distance to the nearest point is less than a set threshold, replacing the coordinates of that point on one line with the coordinates of the nearest point on the other line.

[0050] S600: Traverse each matching combination and merge the two line segments in each matching combination into one line segment.

[0051] In one embodiment, see Figure 2 As shown in step three, merging two line segments into one involves: calculating the lengths of the two line segments along the y-axis, designating the longer segment as the "long line" and the shorter segment as the "short line"; first, removing points between adjacent points of the long line from the short line to obtain the first processed line segment; then, removing points between adjacent points of the first processed line segment from the long line to obtain the second processed line segment; finally, merging the first and second processed line segments and sorting them by their y-coordinates to generate a merged line segment. Adding the merged line segment to the merged line segment list yields the final result.

[0052] The specific implementation method of this application cleverly optimizes the utilization of lane line data, not only maximizing the use of lane line data in each frame of the image, but also reducing the interference of redundant information, making the final obtained lane line information purer and more valuable.

[0053] This invention acquires road image data from multi-view vehicle-mounted cameras in real time as vehicles travel on the road, and annotates road lane lines on all viewpoints to obtain lane line image annotation data. The lane line image annotation data is then read, and the coordinates of each lane line are converted to a LiDAR coordinate system. In the LiDAR coordinate system, the intersection point of the line or its extension on the X-axis, and the angle between the line or its extension and the positive X-axis are calculated for each lane line. Based on the intersection point and angle, a sliding window algorithm is used to perform lane matching for double yellow lines, and then for the remaining single lane lines. Lane matching; merging adjacent line segments of two lines in a matching combination using a nearest neighbor fusion algorithm; traversing each matching combination and merging the two line segments in each matching combination into one line segment; after converting the labeled data to the radar coordinate system, the lane lines in the radar coordinate system are fused and merged, which can effectively improve the matching accuracy and efficiency of lane lines between multiple projections in the radar system, and improve the accuracy and robustness of multi-view lane line fusion technology. The application of this method can significantly simplify the data labeling process, greatly reduce the burden on technical personnel, and free them from this highly repetitive and labor-intensive work.

[0054] This invention also provides an electronic device. Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement any of the lane line marking data fusion and splicing methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0055] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0056] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0057] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0058] This invention also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps in any of the lane line marking data fusion and stitching methods described in the above embodiments. The computer-readable storage medium can be volatile or non-volatile.

[0059] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described lane line marking data fusion and splicing method.

[0060] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0061] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0062] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0063] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0064] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0065] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0066] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0067] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0068] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0069] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A lane line marking data fusion splicing method, characterized in that, The method comprises the following steps: S100, acquiring road image data information captured by a multi-view vehicle-mounted camera when a vehicle travels on a road in real time, and marking road lane lines of all view angles to obtain lane line image marking data information; S200, reading the lane line image marking data and converting point coordinates of each lane line to a laser radar coordinate system; S300, calculating, in the laser radar coordinate system, intersection points of lines or extensions of the lines on an X axis for each lane line and angles of intersection of the lines or extensions of the lines with a positive direction of the X axis; S400, performing lane matching on double-yellow lines by using a sliding window algorithm according to the intersection points and the angles, and performing lane matching on the remaining single lane lines; S500, merging similar line segments of two lines in a matching combination by using a nearby point fusion algorithm; S600, merging two line segments in each matching combination into one line segment by traversing each matching combination.

2. The method of claim 1, wherein, In the step S100, the road image data information is acquired in real time based on a front-view and a surround-view six-view vehicle-mounted camera.

3. The method of claim 1, wherein, In the step S100, the labelme tool is used to mark the road lane lines to obtain lane line image marking data information in a labelme format.

4. The method of claim 1, wherein, In the step S300, intersection points of lines or extensions of the lines on an X axis and angles of intersection of the lines or extensions of the lines with a positive direction of the X axis are calculated for each lane line in the direction of the X axis to obtain a list of intersection point coordinates and a list of angles.

5. The method of claim 1, wherein, In the step S400, the lane matching on the double-yellow lines by using the sliding window algorithm comprises: taking the list of intersection point coordinates and the list of angles as inputs, finding pairs of elements with close intersection point coordinates and angles from the input data by using the sliding window algorithm, and matching the corresponding lane lines as pairs of combinations of the double-yellow lines.

6. The method of claim 5, wherein, The window comprises a plurality of continuous data points, the window is slid to the right successively, the difference between the maximum value and the minimum value in the intersection point coordinate window and the angle window is calculated respectively, if the intersection point coordinate difference and the angle difference are less than preset threshold values, it is determined that the four lines comprise pairs of combinations of the double-yellow lines, and then the pairing information of the double-yellow lines is further determined from the window by the following operations: finding the minimum value in the intersection point coordinate window, then finding elements with the closest difference, taking indexes of the two elements in the original list of intersection point coordinates as a key-value pair and putting the key-value pair into a matching dictionary, and then putting indexes of the remaining two elements into the matching dictionary.

7. The method of claim 5, wherein, In the step S400, the lane matching on the remaining single lane lines comprises: traversing lane lines other than the double-yellow lines to find curves (not including themselves) with the closest intersection point coordinates and saving matching information; if two lines are the closest to each other and the intersection point coordinate difference and the angle difference are less than preset threshold values, the two lines are paired and added to the matching dictionary.

8. The method of claim 1, wherein, The step S500 comprises: traversing each matching combination in the matching dictionary, for two lines in the combination, calculating the distance between each point on one line and each point on the other line, finding the corresponding nearest point on the other line for each point on one line, and replacing the coordinate of the point on one line with the coordinate of the nearest point on the other line if the distance to the nearest point is less than a set threshold.

9. The method of claim 1, wherein, In the step S600, merging the two line segments into one line segment comprises: calculating the length of the two line segments in the y-axis direction, the longer line segment being a long line and the shorter line segment being a short line; first removing the points between the adjacent points of the long line from the short line to obtain a first processed line segment; then removing the points between the adjacent points of the first processed line segment from the long line to obtain a second processed line segment; and finally merging the first processed line segment and the second processed line segment and sorting them according to the y coordinate to generate the merged line segment.

10. An electronic device, comprising: comprise: one or more processors; memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 9.

11. A computer readable medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 9.

12. A computer program product comprising computer readable code, characterized in that, when the computer readable code is running in the processor of the electronic device, the processor in the electronic device performs the method of any one of claims 1 to 9.

Images