Lane line grouping method and device, electronic equipment and computer readable storage medium

Abstract

The invention relates to a lane line grouping method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: based on obtained lane line contour points in the same frame of image, splitting the contour of a lane line to obtain a plurality of convex polygons; the convex polygon is fitted into a rectangle, and the long side direction of the rectangle is the same as the driving direction; performing intra-frame grouping on all the rectangles obtained by fitting; and according to the sequence, carrying out inter-frame combination on the same group of different frames to obtain grouped lane lines. According to the scheme provided by the invention, the lane lines, especially crossed and bent lane lines, can be quickly and efficiently grouped.

Description

Technical Field

[0001] The present application relates to the technical field of high-precision map production for autonomous driving, and in particular to a lane grouping method, device, electronic device and computer-readable storage medium based on image processing technology. Background Art

[0002] The production of high-precision maps requires the processing of large amounts of complex lane line data. Processing all lane line contour points simultaneously may consume a lot of time and computing resources. Therefore, group processing can be used to improve processing efficiency, and grouping can also better adapt to complex road environments to provide more accurate lane line information for autonomous driving.

[0003] In the related technology, clustering methods can be used to group lane line data. However, such methods have high computational complexity, especially for a large number of lane line contour points. They may be inefficient when processing a large number of curved lane line contour points, and may even incorrectly group lane line data collected when the vehicle changes lanes, thereby affecting subsequent lane line processing. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a lane line grouping method, device, electronic device and computer-readable storage medium, which can quickly and efficiently group lane lines, especially intersecting and curved lane lines.

[0005] The first aspect of the present application provides a lane grouping method, comprising: Based on the lane line contour points acquired in the same frame image, the lane line contour is split to obtain multiple convex polygons; Fitting the convex polygon into a rectangle, wherein the long side direction of the rectangle is the same as the driving direction; Grouping all the rectangles obtained by fitting within a frame; In sequence, the same groups of different frames are merged between frames.

[0006] In an optional implementation, the lane line contour is split based on the acquired lane line contour points of the same frame to obtain multiple convex polygons, including: The concave polygon formed by the outline of the intersecting or curved lane line is split into multiple convex polygons, wherein each of the convex polygons represents a different part of the lane line.

[0007] In an optional implementation, the intra-frame grouping of all the rectangles obtained by fitting includes: It is determined whether the horizontal distance between the two rectangles is below a set first distance threshold. If so, the two rectangles are in the same group; otherwise, the two rectangles are in different groups.

[0008] In an optional implementation, the intra-frame grouping of all the fitted rectangles further includes: Sort the grouped rectangles.

[0009] In an optional implementation, the performing inter-frame merging on the same group of different frames in sequence includes: According to different frames, two layers of loops are used to compare and merge the same groups of consecutive frames.

[0010] In an optional implementation, the comparing and merging of the same groups of consecutive frames using two-layer loops according to different frames includes: After determining that the rectangle of the jth group of the i+1th frame and the rectangle of the jth group of the i-th frame have an overlapping area, determining that the horizontal and vertical distances between the rectangle of the jth group of the i+1th frame and the rectangle of the jth group of the i-th frame meet the set conditions; Among them, i and j are both positive integers, the horizontal distance between the rectangle of the jth group in the i+1th frame and the rectangle of the jth group in the i-th frame is below the set second distance threshold, and the angle in the long side direction is below the set angle threshold, and the longitudinal distance is below the set third distance threshold.

[0011] In an optional implementation, the comparing and merging the same groups of consecutive frames using two-layer loops according to different frames further includes: After obtaining the jth merged group according to the matrix of the i-th frame, estimating the rectangle of the jth group of the i+1th frame according to the curvature of the end of the jth merged group, wherein i and j are both positive integers and i≥2; Based on the estimated rectangle, determine whether to merge the rectangles of the jth group in the i+1th frame.

[0012] A second aspect of the present application provides a lane grouping device, comprising: A splitting module is configured to split the contour of the lane line based on the acquired lane line contour points in the same frame image to obtain a plurality of convex polygons; A fitting module, configured to fit the convex polygon into a rectangle, wherein the long side direction of the rectangle is the same as the driving direction; A grouping module, configured to perform intra-frame grouping on all the rectangles obtained by fitting; The merging module is configured to perform inter-frame merging on the same group of different frames in sequence.

[0013] A third aspect of the present application provides an electronic device, including: Processor; and The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0014] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0015] The technical solution provided by this application may have the following beneficial effects: The technical solution of the present application is to split the contour of the lane line corresponding to the lane line contour point in the same frame image to obtain multiple convex polygons, and then fit the convex polygons into rectangles. In the present application, the lane line splitting can clearly identify the lane lines of each part, especially for the crossed or curved lane lines, which can improve the recognition and positioning accuracy of the lane lines, and the splitting and fitting into rectangles, the long side direction of the rectangle is the same as the driving direction, and the direction and boundary of the lane line can also be accurately determined. At the same time, the use of rectangles to fit the lane line contour can simplify the complex lane line shape into a rectangle with clear set characteristics, which is also more convenient for storage and processing; all the rectangles obtained by fitting are grouped within the frame, and the rectangles in the same frame are grouped first, and the amount of data to be grouped is small and it is less likely to have grouping errors; then the same group is merged between frames according to the sequence of different frames to obtain the grouped lane line rectangles. In the present application, the lane line grouping and merging by means of rectangles can improve the processing efficiency of the lane line, make the grouping efficiency higher, and can also reduce the grouping error. Therefore, the technical solution of the present application can quickly and efficiently group the lane lines, especially the crossed and curved lane lines.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0018] Figure 1 is a flowchart of a lane grouping method shown in an embodiment of the present application; Figure 2 is a schematic diagram of the structure of a lane grouping device shown in an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0020] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0022] In the related technology, clustering methods can be used to group lane line data. However, such methods have high computational complexity, especially for a large number of lane line contour points. They may be inefficient when processing a large number of curved lane line contour points, and may even incorrectly group lane line data collected when the vehicle changes lanes, thereby affecting subsequent lane line processing.

[0023] In response to the above problems, an embodiment of the present application provides a lane line grouping method, which can quickly and efficiently group lane lines, especially intersecting and curved lane lines.

[0024] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0025] Figure 1 It is a flowchart of the lane line grouping method shown in an embodiment of the present application.

[0026] like Figure 1 As shown, the embodiment of the present application provides a lane line grouping method, wherein the lane lines are preferably crossed or curved lane lines. The method mainly includes steps S101 to S104.

[0027] Step S101: based on the acquired lane line contour points in the same frame image, the lane line contour is split to obtain a plurality of convex polygons.

[0028] For crossed or curved lane lines, the contours are irregular. Splitting crossed or curved lane lines into multiple lane lines can make each lane line clearer and more specific, thereby avoiding confusion and uncertainty caused by crossed or curved lane lines. Then, step S101 may further include: splitting the concave polygon formed by the contour of crossed or curved lane lines into multiple convex polygons, wherein each convex polygon represents a different part of the lane line. For actual lane lines, there will be concave parts in the contour of the lane lines, that is, the contour of the lane lines can be known from the contour points of the lane lines in the same frame image obtained, and then the concave polygon formed by the contour of the lane lines is split into multiple convex polygons, and each convex polygon can represent a different part of the lane lines.

[0029] Step S102: Fit the convex polygon into a rectangle, wherein the long side direction of the rectangle is the same as the driving direction.

[0030] It is determined from step S101 that each convex polygon can represent different parts of the lane line. In step S102, each convex polygon is fitted to obtain a rectangle corresponding to each convex polygon, wherein the rectangle has length and width, four vertices and 1 center point, and a direction, which is roughly consistent with the driving direction, and the direction is used as the long side direction of the rectangle. When each rectangle has a direction, the direction of the rectangle can be used to provide intuitive direction guidance for vehicle driving. Then, in autonomous driving, the vehicle can more easily adjust the driving direction according to the direction of the rectangle.

[0031] Step S103: grouping all the fitted rectangles within the frame.

[0032] Step S103 may further include determining whether the horizontal distance between the two rectangles is below a set first distance threshold; if so, the two rectangles are in the same group; otherwise, the two rectangles are in separate groups.

[0033] In this embodiment, step S103 determines whether the lateral distance between the two rectangles is below a set first distance threshold, wherein the lateral distance can be calculated by the projection distance of the two rectangle center points in the short side direction of the rectangle, and the first distance threshold L1 is determined according to the lateral distance of the two-line lane line and the lateral distance between two adjacent lane lines in the multi-line lane line.

[0034] When the lane line is a two-line lane line, when the lateral distance between the two rectangles is determined to be below the first distance threshold L1, it can be determined that the lane line corresponding to the two rectangles is a two-line lane line; when the lateral distance between the two rectangles is determined to exceed the first distance threshold L1, it means that the lateral distance is far, and they belong to different lane lines and cannot be processed in one group.

[0035] For lane lines with more than three lines, the lateral distance between any two adjacent lane lines will also satisfy that the lateral distance between the two rectangles is below the set first distance threshold, and then all related multiple rectangles can be divided into one group. Obviously, the judgment result of yes in step S103 has a higher priority.

[0036] Considering that the present application is ultimately to group the entire lane lines, rather than the lane line data in a single frame image, step S103 may further include sorting the grouped rectangles. In this embodiment, the sorting principle may be to assume that the groups of lane lines in each frame are in front of the vehicle, align the timestamps of multiple frames of images, and then sort the lane lines in each frame image from left to right. Among them, for more than two rectangles in the same group, they may also be further sorted from left to right based on the current group.

[0037] Step S104: performing inter-frame merging on the same groups of different frames according to the sequence.

[0038] Step S104 may further include comparing and merging identical groups of consecutive frames using two-layer loops according to different frames.

[0039] In this embodiment, the order of image acquisition is determined according to the timestamps corresponding to different frames of images. Taking the merging of two consecutive frames of images as an example, two layers of loops are used to compare and merge the same groups of the two consecutive frames according to the acquisition time of different frames. The first and second consecutive frames each include several groups. A group of data is taken from all lane contour point data corresponding to the second frame, and all groups in the second frame are traversed through the outer loop. For each group in the second frame, an inner loop is used to compare it with each group in the first frame. After the comparison, the first frame and the second frame are merged, that is, the data of the two groups to be merged are integrated.

[0040] For ease of understanding, for example, if the first frame and the second frame are both two-dimensional arrays, each element can be regarded as a group, then through two layers of loops, each element in the second frame can be taken out in turn and compared with each element in the first frame. If certain conditions are met, they are merged into a new element.

[0041] Specifically, step S104 may include: After determining that the rectangle of the jth group of the i+1th frame and the rectangle of the jth group of the ith frame have an overlapping area, determining that the horizontal and vertical distances between the rectangle of the jth group of the i+1th frame and the rectangle of the jth group of the ith frame meet the set conditions; Among them, i and j are both positive integers, the horizontal distance between the rectangle of the jth group in the i+1th frame and the rectangle of the jth group in the i-th frame is below the set second distance threshold, the angle in the long side direction is below the set angle threshold, and the vertical distance is below the set third distance threshold.

[0042] In this embodiment, the merging principle can be that two rectangles can be merged into one group if they have an overlapping area, and further require that the two rectangles can be merged into one group if the horizontal distance is close and the direction angle is small and close to parallel, and the two rectangles cannot be merged into one group if the vertical distance is far and exceeds the third distance threshold. The merging can be performed only after all the above requirements are met, otherwise it cannot be merged. Assuming that rectangle A of the first group in the second frame is compared and merged with rectangle B of the first group in the first frame, and rectangle A and rectangle B have an overlapping area, it is further judged whether the horizontal distance between rectangle A and rectangle B is close and whether the direction angle is small and close to parallel. Under the premise of both, it is considered whether rectangle A and rectangle B are far apart in the vertical direction. Specifically, it can be quantified by the second distance threshold, the angle threshold and the third distance threshold, wherein the horizontal distance can be calculated by the projection distance of the two rectangle center points of the rectangle in the short side direction of the rectangle, which can also be understood as calculated by the distance of the straight line where the two rectangles are located, and the vertical distance can be calculated by the projection distance of the two rectangle center points of the two rectangles in the long side direction.

[0043] Specifically, step S104 may also include: After obtaining the jth merged group according to the matrix of the ith frame, the rectangle of the jth group of the i+1th frame is estimated according to the curvature of the end of the jth merged group, where i and j are both positive integers and i≥2; Based on the estimated rectangle, determine whether to merge the rectangles of the jth group in the i+1th frame.

[0044] In this embodiment, multiple merge groups can be obtained from step S103. Assuming that after the 1st to 4th frames are merged, 3 merge groups are obtained. Taking the first group of the three merge groups as an example, the curvature of the terminal curve of the merge group is within a certain range. It can be determined whether the new frame rectangle to be added to the merge group can be merged into the group. Specifically, the curvature of the terminal curve of the merge group can be used to estimate the estimated rectangle of the merge group at the distance where the new frame rectangle is located, and then it can be determined whether the estimated rectangle and the new frame rectangle can be merged. This embodiment can be used as a restriction condition to further determine whether the new frame rectangle can be merged into the merge group, which can improve the quality of the merge after grouping, and thus can make the final grouping effect of the lane line better.

[0045] It can be seen that the embodiment of the present application obtains multiple convex polygons by splitting the concave polygons of the lane line contour corresponding to the lane line contour points in the same frame image, and then fitting the convex polygons into rectangles. Splitting the lane lines can clearly identify the lane lines in each part, especially for crossed or curved lane lines, which can improve the recognition and positioning accuracy of the lane lines. The splitting and fitting are rectangles, and the long side direction of the rectangle is the same as the driving direction, which can also accurately determine the direction and boundary of the lane line. After that, all the fitted rectangles are grouped within the frame, and the same group is merged between frames according to the time sequence of different frames to obtain the grouped lane line rectangles. In the present application, the lane lines are grouped and merged with the help of rectangles, which can improve the processing efficiency of the lane lines and make the grouping efficiency higher.

[0046] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a lane line grouping device, an electronic device and corresponding embodiments.

[0047] Figure 2 A schematic diagram of the structure of a lane grouping device shown in an embodiment of the present application.

[0048] like Figure 2 As shown, an embodiment of the present application provides a lane line grouping device, including: A splitting module 201 is configured to split the contour of the lane line based on the acquired lane line contour points in the same frame image to obtain a plurality of convex polygons; A fitting module 202, configured to fit the convex polygon into a rectangle, wherein the long side direction of the rectangle is the same as the driving direction; A grouping module 203, which is configured to perform intra-frame grouping on all fitted rectangles; The merging module 204 is configured to perform inter-frame merging on the same group of different frames in sequence.

[0049] In some embodiments, the splitting module 201 may be further configured to split the concave polygon formed by the outline of the crossing or curved lane line into a plurality of convex polygons, wherein each of the convex polygons represents a different portion of the lane line.

[0050] In some embodiments, the grouping module 203 may be further configured to determine whether the horizontal distance between the two rectangles is below a set first distance threshold; if so, the two rectangles are in the same group; otherwise, the two rectangles are in separate groups.

[0051] In some embodiments, the grouping module 203 may be further configured to sort the grouped rectangles.

[0052] In some embodiments, the merging module 204 may be further configured to compare and merge the same groups of consecutive frames using a two-layer loop according to different frames.

[0053] In at least one embodiment, the merging module 204 may be configured to determine that, after determining that the rectangle of the jth group of the i+1th frame and the rectangle of the jth group of the i-th frame have an overlapping area, determine that the horizontal and vertical distances between the rectangle of the jth group of the i+1th frame and the rectangle of the jth group of the i-th frame meet a set condition; Among them, i and j are both positive integers, the horizontal distance between the rectangle of the jth group in the i+1th frame and the rectangle of the jth group in the i-th frame is below the set second distance threshold, and the angle in the long side direction is below the set angle threshold, and the longitudinal distance is below the set third distance threshold.

[0054] In at least one embodiment, the merging module 204 may also be configured to estimate the rectangle of the jth group of the i+1th frame according to the curvature of the end of the jth merging group after obtaining the jth merging group according to the matrix of the ith frame, wherein i and j are both positive integers and i≥2; Based on the estimated rectangle, determine whether to merge the rectangles of the jth group in the i+1th frame.

[0055] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0056] Figure 3 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0057] See also Figure 3 , the electronic device 300 includes a memory 301 and a processor 302.

[0058] The processor 302 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0059] The memory 301 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 302 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 301 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 301 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0060] The memory 301 stores executable codes, and when the executable codes are processed by the processor 302 , the processor 302 can execute part or all of the methods described above.

[0061] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0062] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0063] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A lane line grouping method, characterized in that: include: Based on the lane line contour points acquired in the same frame image, the lane line contour is split to obtain multiple convex polygons; Fitting the convex polygon into a rectangle, wherein the long side direction of the rectangle is the same as the driving direction; Grouping all the rectangles obtained by fitting within a frame; In sequence, the same groups of different frames are merged between frames.

2. The lane grouping method according to claim 1, characterized in that: Based on the acquired lane line contour points of the same frame, the lane line contour is split to obtain multiple convex polygons, including: The concave polygon formed by the outline of the intersecting or curved lane line is split into multiple convex polygons, wherein each of the convex polygons represents a different part of the lane line.

3. The lane grouping method according to claim 1, characterized in that: The intra-frame grouping of all the rectangles obtained by fitting includes: It is determined whether the horizontal distance between the two rectangles is below a set first distance threshold. If so, the two rectangles are in the same group; otherwise, the two rectangles are in different groups.

4. The lane grouping method according to claim 3, characterized in that: The intra-frame grouping of all the fitted rectangles further includes: Sort the grouped rectangles.

5. The lane grouping method according to claim 1, characterized in that: The step of performing inter-frame merging on the same group of different frames in sequence includes: According to different frames, two layers of loops are used to compare and merge the same groups of consecutive frames.

6. The lane grouping method according to claim 5, characterized in that: The method uses two layers of loops to compare and merge the same groups of consecutive frames according to different frames, including: After determining that the rectangle of the jth group of the i+1th frame and the rectangle of the jth group of the i-th frame have an overlapping area, determining that the horizontal and vertical distances between the rectangle of the jth group of the i+1th frame and the rectangle of the jth group of the i-th frame meet the set conditions; Among them, i and j are both positive integers, the horizontal distance between the rectangle of the jth group in the i+1th frame and the rectangle of the jth group in the i-th frame is below the set second distance threshold, and the angle in the long side direction is below the set angle threshold, and the longitudinal distance is below the set third distance threshold.

7. The lane grouping method according to claim 6, characterized in that: The method of comparing and merging the same groups of consecutive frames using two-layer loops according to different frames also includes: After obtaining the jth merged group according to the matrix of the i-th frame, estimating the rectangle of the jth group of the i+1th frame according to the curvature of the end of the jth merged group, wherein i and j are both positive integers and i≥2; Based on the estimated rectangle, determine whether to merge the rectangles of the jth group in the i+1th frame.

8. A lane grouping device, characterized in that: include: A splitting module is configured to split the contour of the lane line based on the acquired lane line contour points in the same frame image to obtain a plurality of convex polygons; A fitting module, configured to fit the convex polygon into a rectangle, wherein the long side direction of the rectangle is the same as the driving direction; A grouping module, configured to perform intra-frame grouping on all the rectangles obtained by fitting; The merging module is configured to perform inter-frame merging on the same group of different frames in sequence.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: An executable code is stored thereon, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as claimed in any one of claims 1 to 7.

Images