Lane Line Processing Method, Device, Equipment and Storage Medium

Abstract

The present application relates to a lane line processing method, apparatus, device and storage medium. The method includes: generating a first storage list and a second storage list for storing a lane line segment data set; taking the lane line segment line_i in the first storage list as the lane line segment to be processed, and traversing the lane line segments in the first storage list from the (i + 1)-th storage position until the lane line segment line_j that meets the set merging condition is determined; marking the index value of the storage position where the lane line segment line_j' corresponding to line_j in the second storage list is located as i', and linking line_j' to the storage position corresponding to the index value i'; taking the lane line segment line_i+1 as the next lane line segment to be processed until the linking of the lane line segments in the second storage list and the marking of the index values of their storage positions are completed; and outputting the target lane line data according to the lane line segment data set in the second storage list for which the linking and index value marking have been completed. The solution provided by the present application can avoid the problems of incorrect merging and missed merging, and improve the accuracy of lane line merging.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and particularly to a method, apparatus, device and storage medium for lane line processing in the production of electronic maps. Background Art

[0002] Lane lines are the left and right boundary lines that determine the scope of a driving lane. Generally, the lane scope is determined by the lane lines printed on the ground, which are roughly divided into five types: single dotted lines, single solid lines, double solid lines, double dotted lines, and broken and solid lines, to ensure that vehicles drive within the correct lanes and provide safety guarantees for vehicle driving. In the production of high-precision maps for autonomous driving, they are the most important road elements.

[0003] In the related art, a lane line image is obtained, and in the way of instance segmentation, according to the pixel data in the same picture, it is determined whether these pixel data belong to the same lane line. However, this method cannot determine whether the lane line segments in different pictures belong to the same lane line.

[0004] To solve the above problems, generally, according to the distance from the pixel coordinates of the lane line to the vehicle, it is confirmed whether the lane line segments in different pictures belong to the same lane line. However, if the vehicle changes lanes during driving, or there are situations of lane line confluence or divergence, this method cannot accurately distinguish either. Summary of the Invention

[0005] To solve or partially solve the problems existing in the related art, the present application provides a lane line processing method, apparatus, device and storage medium, which can avoid the problems of incorrect merging and missed merging in lane line processing and improve the accuracy of lane line merging.

[0006] The first aspect of the present application provides a lane line processing method, including the following steps:

[0007] S1. Generate a first storage list and a second storage list for storing a lane line segment data set. One storage bit in the first storage list and the second storage list stores one lane line segment data, and the distribution of the lane line segment data in the first storage list and the second storage list is the same;

[0008] S2. Take the lane line segment line_i in the i-th storage bit of the first storage list as the lane line segment to be processed, and traverse the lane line segments in the first storage list from the (i + 1)-th storage bit until the lane line segment line_j in the j-th storage bit that meets the set merging condition with line_i is determined, where j > i, and i = 1, 2, 3,..., n - 1, and n represents the total number of storage bits in the first storage list;

[0009] S3. Mark the index value of the storage location where the lane line segment line_j' corresponding to line_j in the second storage list is located as i', and link line_j' to the storage location corresponding to the index value i', where i' represents the storage location where the lane line segment line_i' corresponding to line_i in the second storage list is located;

[0010] S4. Take the lane line segment line_i+1 in the (i + 1)-th storage location of the first storage list as the next lane line segment to be processed, and repeat S2 and S3 until the linking of the lane line segments in the second storage list and the marking of the index values of their storage locations are completed;

[0011] S5. Output the target lane line data according to the lane line segment data set in the second storage list for which the linking and index value marking have been completed.

[0012] As an optional embodiment, the number of storage locations in the first storage list and the second storage list is not less than the total number of lane line segments.

[0013] As an optional embodiment, the same lane line segment is stored in the same storage location in the first storage list and the second storage list.

[0014] As an optional embodiment, the outputting of the target lane line data according to the lane line segment data set in the second storage list for which the linking and index value marking have been completed includes the following steps:

[0015] Filter and delete the storage locations in the second storage list marked with index values, and use the lane line segment data set in the remaining storage locations as the target lane line data.

[0016] As an optional embodiment, the linking of line_j' to the storage location corresponding to the index value i' includes the following steps:

[0017] If the storage location where the lane line segment line_i' corresponding to line_i in the second storage list is located is not marked with an index value, directly link line_j' to the storage location corresponding to the index value i';

[0018] If the storage location where the lane line segment line_i' corresponding to line_i in the second storage list is located is marked with an index value, link line_j' to the storage location corresponding to the index value of the storage location where line_i' is located.

[0019] As an optional embodiment, taking the lane segment line_i in the i-th storage bit of the first storage list as the lane segment to be processed, traversing the lane segments in the first storage list from the (i + 1)-th storage bit until determining the lane segment line_j in the j-th storage bit that meets the set merging condition with line_i, includes the following steps:

[0020] Extend at least one end of the current lane segment line_i to be processed in the first storage list by a first set length to obtain the extended line line_e of at least one end of line_i;

[0021] Traverse the lane segments in the first storage list from the (i + 1)-th storage bit, determine the lane segments whose distance from line_e is less than the set threshold, and take them as the lane segments to be screened;

[0022] Extend at least one end of the lane segment to be screened by a second set length to obtain the extended line line_e' of at least one end of the lane segment to be screened;

[0023] Calculate the included angle α between line_e' and line_e. If α is less than the set angle, take the lane segment to be screened as the lane segment line_j.

[0024] As an optional embodiment, the step of extending at least one end of the current lane segment line_i to be processed in the first storage list by a first set length to obtain the extended line line_e of at least one end of line_i includes the following steps:

[0025] According to the slope of the current lane segment line_i to be processed in the first storage list, select the first reference points at the head and tail ends of line_i, and extend both ends of line_i by the first set length based on the first reference points to obtain the extended lines line_e of both ends of line_i 1 and line_e 2 .

[0026] As an optional embodiment, the step of extending at least one end of the lane segment to be screened by a second set length to obtain the extended line line_e' of at least one end of the lane segment to be screened includes the following steps:

[0027] According to the slope of the lane segment to be screened, select the second reference points at the head and tail ends of the lane segment to be screened, and extend both ends of the lane segment to be screened by the second set length based on the second reference points to obtain the extended lines line_e 1 ' and line_e 2 '.

[0028] As an alternative embodiment, traversing the lane segments in the first storage list from the (i + 1)-th storage bit, determining the lane segments whose distance from line_e is less than a set threshold, and taking them as the lane segments to be screened, includes the following steps:

[0029] Traverse the lane segments in the first storage list from the (i + 1)-th storage bit, and calculate the shortest distance D between each lane segment and line_e 1 and the shortest distance D between each lane segment and line_e 1 ; 2 and the shortest distance D between each lane segment and line_e 2 ;

[0030] Compare the smaller value between D 1 and D 2 with the set threshold;

[0031] If the smaller value between D 1 and D 2 is less than the set threshold, then take this lane segment as the lane segment to be screened.

[0032] As an alternative embodiment, before calculating the included angle α between line_e' and line_e, includes the following steps:

[0033] Calculate the shortest distance D 1 ' between line_e 2 ' and line_e 1 and the shortest distance D 1 '' between line_e 1 ' and line_e 1 ', and the shortest distance D 2 ' between line_e 2 ' and line_e 2 ' and the shortest distance D 2 '' between line_e

[0034] Select the extension line corresponding to the smaller value between D 1 ' and D 1 '' as the first extension line to be screened;

[0035] Select the extension line corresponding to the smaller value between D 2 ' and D 2 '' as the second extension line to be screened.

[0036] As an alternative embodiment, when calculating the included angle α between line_e' and line_e, if α is less than the set angle, then taking the lane segment to be screened as the lane segment line_j, includes the following steps:

[0037] Calculate the included angle α between the first extension line to be screened and line_e 1 and the included angle α between the second extension line to be screened and line_e 1 ; 2 ; 2 ;

[0038] If α 1 and / or α 2 is less than the set angle, then use the lane line segment to be screened as the lane line segment line_j.

[0039] The second aspect of the present application provides a lane line processing device, including:

[0040] A generation module, configured to generate a first storage list and a second storage list for storing a lane line segment data set. The number of storage bits in the first storage list and the second storage list is not less than the total amount of lane line segments, and one storage bit stores one lane line segment data; and the distribution of lane line segment data in the first storage list and the second storage list is the same;

[0041] A calculation module, configured to use the lane line segment line_i in the i-th storage bit of the first storage list as the lane line segment to be processed, traverse the lane line segments in the first storage list from the (i + 1)-th storage bit until the lane line segment line_j in the j-th storage bit that satisfies the set merging condition with line_i is determined, where j > i, and i = 1, 2, 3,..., n - 1, and n represents the total number of storage bits storing lane line segment data in the first storage list;

[0042] A marking module, configured to mark the index value of the storage bit where the lane line segment line_j' corresponding to line_j in the second storage list is located as i', and link line_j' to the storage bit corresponding to the index value i', where i' represents the storage bit where the lane line segment line_i' corresponding to line_i in the second storage list is located; the calculation module is further configured to use the lane line segment line_i+1 in the (i + 1)-th storage bit of the first storage list as the next lane line segment to be processed, repeat traversing and calculating until the linking of the lane line segments in the second storage list and the marking of the index values of their storage bits are completed;

[0043] An output module, configured to output target lane line data according to the lane line segment data set in the second storage list that has completed linking and index value marking.

[0044] The third aspect of the present application provides an electronic device, including:

[0045] A processor; and

[0046] A memory stores executable code thereon, which, when executed by the processor, causes the processor to execute the method as described above.

[0047] The fourth aspect of the present application provides a computer-readable storage medium storing executable code thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method as described above.

[0048] The technical solution provided by the present application may include the following beneficial effects:

[0049] By generating a first storage list storing a set of lane line segments and a second storage list matching the first storage list, in the process of processing lane lines, it is determined whether the lane line segment data in the first storage list meets the merging conditions, and on the basis of not changing the lane line segment data stored in the first storage list, the lane line segment data that can be merged in the second storage list is marked with an index value by comparing with the first storage list, and then the position where the lane line segment data that can be merged is located is accurately located according to the index value, so as to realize the fast merging process of the corresponding lane line segments, and avoid the mis-merging and missed-merging of lane line segments.

[0050] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] By describing the exemplary embodiments of the present application in more detail in conjunction with the drawings, the above and other objects, features, and advantages of the present application will become more obvious, wherein, in the exemplary embodiments of the present application, the same reference numerals generally represent the same components.

[0052] Figure 1 is a flowchart of the lane line processing method shown in the embodiments of the present application;

[0053] Figure 2 is a schematic structural diagram of the first storage list shown in the embodiments of the present application;

[0054] FIG. 3(a) is a schematic structural diagram of the initial style of the second storage list shown in the embodiments of the present application;

[0055] FIG. 3(b) is a schematic structural diagram of the second storage list shown in FIG. 3(a) after being marked;

[0056] FIG. 3(c) is a schematic structural diagram of the second storage list shown in FIG. 3(b) after being merged;

[0057] Figure 4 is another schematic structural diagram of the first storage list shown in the embodiments of the present application;

[0058] FIG. 5(a) is another schematic structural diagram of the initial style of the second storage list shown in the embodiment of the present application;

[0059] FIG. 5(b) is a schematic structural diagram of the second storage list with marks completed as shown in FIG. 5(a);

[0060] FIG. 5(c) is a schematic structural diagram of the second storage list with merging completed as shown in FIG. 5(b);

[0061] Figure 6 is a schematic structural diagram of the lane line processing device shown in the embodiment of the present application;

[0062] Figure 7 is a schematic structural diagram of the electronic device shown in the embodiment of the present application. Detailed implementation manners

[0063] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present application will be more thorough and complete, and can fully convey the scope of the present application to those skilled in the art.

[0064] The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "the" and "said" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0065] It should be understood that although the terms "first", "second", "third", etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, "a plurality" means two or more unless otherwise specifically defined.

[0066] In the related art, to obtain a lane line image, instance segmentation is used to determine whether pixel data in the same picture belongs to the same lane line according to the pixel data. However, this method cannot determine whether lane line segments in different pictures belong to the same lane line. To solve the above problem, generally, it is determined whether lane line segments in different pictures belong to the same lane line according to the distance from the pixel coordinates of the lane line to the vehicle. However, if the vehicle changes lanes during driving, or there are situations of lane line merging or splitting, this method cannot accurately distinguish them either.

[0067] In view of the above problems, an embodiment of the present application provides a lane line processing method, which can avoid the problems of incorrect merging and missed merging in lane line processing and improve the accuracy of lane line merging.

[0068] The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0069] Figure 1 It is a schematic flowchart of the lane line processing method shown in the embodiment of the present application.

[0070] See Figure 1 , an embodiment of the present application provides a lane line processing method, including the following steps S1 to S5:

[0071] S1. Generate a first storage list and a second storage list for storing a lane line segment data set, and one storage bit in the first storage list and the second storage list stores one lane line segment data.

[0072] The lane line segment data set in the embodiment of the present application may refer to multiple lane line segment data obtained by distance clustering of lane line pixel coordinate points in multiple frames of images, where one lane line segment data is obtained by clustering each frame of image.

[0073] The embodiment of the present application can store the above-mentioned multiple lane line segment data in multiple storage bits of the first storage list and the second storage list. Among them, the style of the first storage list can be seen in Figure 2 .

[0074] The distribution of the lane line segment data in the first storage list and the second storage list is the same. For example, it can be concentrated or discontinuous. The embodiment of the present application preferably has a concentrated distribution.

[0075] Specifically, the same storage bit in the first storage list and the second storage list stores the same lane line segment data, or the relative position relationship of the storage bits where each lane line segment data is located in the first storage list is the same as that in the second storage list, so that the storage bits where the lane line segment data in the first storage list is located can be mapped to the storage bits where the lane line segment data in the second storage list is located.

[0076] In the embodiment of the present application, the initial style (which may refer to the data distribution method) of the second storage list can be exactly the same as that of the first storage list. For the style of the first storage list, reference can be made to Figure 2 , and for the initial style of the second storage list, reference can be made to Fig. 3(a). During the lane line processing, the data in the first storage list can always remain unchanged, and the data in the first storage list is mainly used for the calculation of merging conditions, and its style remains the same Figure 2 . However, the data in the second storage list will change according to the calculation result of the first storage list. Exemplarily, the second storage list can mark the index value according to the calculation result of the first storage list, and link the lane segment data to the corresponding storage bit according to the index value (here, "link" can be understood as moving (or merging) the lane segment data to the storage bit corresponding to its index value). Therefore, the lane line data in the storage bit of the second storage list is always changing. Through the cooperation of the two storage lists in the embodiment of the present application, it is possible to neither change the lane segment data used for merging calculation nor mark and merge the lane segment data that meets the merging conditions, which can greatly reduce the data processing volume and improve the processing efficiency.

[0077] Preferably, the number of storage bits of the first storage list and the second storage list is not less than the total amount of lane segments. For example, n lane line data are placed in the first storage list and the second storage list with at least n storage bits. The first storage bit storing lane segment data is regarded as the 1st storage bit, the i-th storage bit storing lane segment data is regarded as the i-th storage bit, the j-th storage bit storing lane segment data is regarded as the j-th storage bit, and the n-th storage bit storing lane segment data is regarded as the n-th storage bit.

[0078] Further preferably, the number of storage bits of the first storage list and the second storage list is the same as the total amount of lane segments.

[0079] In the embodiment of the present application, the number of storage bits of the first storage list and the second storage list can be set according to the total amount of actual lane segment data, so that each storage bit of the first storage list and the second storage list stores one lane segment data, which can further reduce the calculation amount.

[0080] Preferably, the same storage bit in the first storage list and the second storage list stores the same lane segment.

[0081] In the embodiment of the present application, the second storage list can also be obtained by copying the first storage list, so that each same storage bit in the first storage list and the second storage list stores the same lane segment, which can avoid storing the huge amount of lane line data twice repeatedly.

[0082] S2. Take the lane line segment line_i in the i-th storage bit of the first storage list as the lane line segment to be processed, and traverse the lane line segments in the first storage list from the (i + 1)-th storage bit until the lane line segment line_j in the j-th storage bit that meets the set merging condition with line_i is determined, where j > i, and i = 1, 2, 3,..., n - 1, and n represents the total number of storage bits storing lane line segment data in the first storage list.

[0083] The initial lane line segment to be processed in the embodiment of the present application can be arbitrarily selected from the first storage list, preferably the lane line segment line_1 in the first storage bit.

[0084] Based on the lane line segment line_i in the i-th storage bit, when traversing other lane line segments in the first storage list in the embodiment of the present application, start traversing from the (i + 1)-th storage bit from the back instead of from the first bit to the back, and only traverse the storage bits greater than i, which can reduce the number of traversals and improve the processing efficiency of lane lines.

[0085] After the embodiment of the present application determines the lane line segment line_j in the j-th storage bit that meets the set merging condition with line_i, then take the lane line segment in the (i + 1)-th storage bit as the next lane line segment to be processed, which can avoid missing lane line segment data. And from i to n - 1, that is, the last storage bit (the n-th storage bit) storing lane line segment data is not subjected to merging calculation. This is because the lane line segment data in the n-th storage bit has two possibilities. One is that it already has an index value, and the other is that it does not meet the merging condition with the lane line segment data from the i-th storage bit to the n-th storage bit. Therefore, there is no need to perform merging calculation on the n-th storage bit, which can further save the calculation amount.

[0086] The embodiment of the present application can determine the lane line segment line_j in the j-th storage bit that meets the set merging condition with the lane line segment line_i in the i-th storage bit in the following way: Extend at least one end of the current lane line segment line_i in the first storage list by a first set length to obtain the extended line line_e of at least one end of line_i; Traverse the lane line segments in the first storage list from the (i + 1)-th storage bit, and determine the lane line segments whose distance from line_e is less than the set threshold and use them as the lane line segments to be screened; Extend at least one end of the lane line segment to be screened by a second set length to obtain the extended line line_e' of at least one end of the lane line segment to be screened; Calculate the included angle α between line_e' and line_e. If α is less than the set angle, then take the lane line segment to be screened as the lane line segment line_j.

[0087] The embodiments of the present application determine whether the set merging conditions are met from multiple aspects such as the distance and angle between the extension lines of two lane line segments, which can reduce the mis-merging and missed-merging of lane line segments.

[0088] S3. Mark the index value of the storage location corresponding to the lane line segment line_j' of line_j in the second storage list as i', and link line_j' to the storage location corresponding to the index value i', where i' represents the storage location corresponding to the lane line segment line_i' of line_i in the second storage list.

[0089] In the embodiments of the present application, the storage location corresponding to the lane line segment line_j' corresponding to the j-th storage location in the first storage list can be determined from the second storage list by referring to the distribution of the storage locations storing the lane line segment data in the first storage list, as well as the storage location corresponding to the lane line segment line_i' corresponding to the lane line segment line_i in the i-th storage location in the first storage list. Since the distribution of the lane line segment data in the two storage lists is the same or mapped, the data of the lane line segment line_j in the j-th storage location in the first storage list is the same as the data of the lane line segment line_j' in the j'-th storage location in the second storage list, and the data of the lane line segment line_i in the i-th storage location in the first storage list is the same as the data of the lane line segment line_i' in the i'-th storage location in the second storage list. Therefore, the storage location storing the same lane line segment data in the second storage list can be determined according to the positions of line_i and line_j in the first storage list. Among them, i and i', as well as j and j', can be the same or mapped in value. For example, both i and i' are 1 or both refer to the first storage location, and both j and j' are 3 or both refer to the third storage location.

[0090] Specifically, referring to FIG. 3(b), the embodiments of the present application mark the index value of the storage location corresponding to the lane line segment line_j' of line_j in the second storage list as i'. According to the index value i' marked by the storage location where line_j' is located, it can be determined that the storage location of the lane line segment that meets the set merging conditions with line_j' is the i'-th storage location, and the lane line segment stored in the i'-th storage location is line_i'; referring to FIG. 3(c), then link the lane line segment line_j' to the i'-th storage location to realize the data merging of line_j' and line_i'.

[0091] Preferably, if the storage position corresponding to the lane line segment line_i' of line_i in the second storage list is not marked with an index value, it indicates that the lane line segment before the storage position of the lane line segment line_i' does not meet the set merging condition with the lane line segment line_i', or it indicates that the lane line segment before the storage position of the lane line segment line_i' has not been calculated with the lane line segment line_i'. Then, line_j' can be directly linked to the storage position corresponding to the index value i'.

[0092] If the storage position corresponding to the lane line segment line_i' of line_i in the second storage list is marked with an index value, it indicates that the lane line segment before the storage position of the lane line segment line_i' meets the set merging condition with the lane line segment line_i', and the lane line segment line_i' has been linked to the storage position corresponding to its index value. Then, the lane line segment line_j' is also linked to the storage position corresponding to the index value of the storage position where line_i' is located.

[0093] S4. Take the lane line segment line_i+1 in the (i + 1)-th storage position of the first storage list as the next lane line segment to be processed, and repeat S2 and S3 until the linking of the lane line segments in the second storage list and the marking of the index values of their storage positions are completed.

[0094] Taking the lane line segment line_i+1 in the (i + 1)-th storage position of the first storage list as the next lane line segment to be processed and repeating the above steps S2 and S3 can complete the merging calculation for the storage positions from the i-th storage position to the (n - 1)-th storage position in the first storage list, improving the comprehensiveness of the calculation.

[0095] S5. Generate target lane line data according to the lane line segment data set in the second storage list for which the linking and index value marking have been completed.

[0096] Based on the above steps, after the linking of the lane line segments in the second storage list and the marking of their index values are completed, the storage positions corresponding to the lane line segments that can be merged in the second storage list are all marked with index values, and the lane line segments marked with index values have all been linked to the storage positions corresponding to the index values. The merged lane line segment data can be obtained according to the linking and indexing results.

[0097] In the embodiment of the present application, by generating a first storage list and a second storage list for storing a set of lane line segments, during the process of processing lane lines, the lane line segment data in the first storage list remains unchanged. The lane line segment data in the first storage list is calculated for the merging condition, and then the lane line segment data that can be merged in the second storage list is marked with an index value with reference to the first storage list. And according to the index value, the lane line segment with a later storage position is accurately linked to the storage position where the lane line segment that can be merged and has an earlier storage position is located, so as to realize the fast merging process of the corresponding lane line segments, and can avoid the incorrect merging and missed merging of lane line segments.

[0098] As an optional embodiment, step S5 outputs the target lane line data according to the lane line segment data set in the second storage list that has been completed with link and index value marking, including the following steps:

[0099] Filter and delete the storage positions marked with index values in the second storage list, and take the lane line segment data set in the remaining storage positions as the target lane line data and output it.

[0100] In the embodiment of the present application, the lane line segment data in the storage positions that have been completed with index value marking in the second storage list has been linked to the corresponding storage positions. Therefore, the values in the storage positions that have been completed with index value marking are set values, such as 1. When outputting the target lane line data, filter and delete the positions with set values, and the remaining storage positions are several lane line segment data merged together. Refer to Fig. 3(c). In the figure, the values of the storage positions where the lane line segments marked with index values are located are all set to set values.

[0101] Since the lane line segments that have been completed with marking are all linked to the positions where the lane line segments corresponding to the index values are located, then the values of the positions where the lane line segments that have been completed with marking are located are all set values, such as 1. In the subsequent next cycle process, these positions with set values of 1 can be filtered out, and the cycle is repeated from the positions where the set values are not 1, so as to avoid redundant calculations.

[0102] As an optional embodiment, in step S2, to determine the currently to-be-processed lane line segment line_i in the first storage list, traverse the lane line segments in the first storage list from the (i + 1)-th storage position until the lane line segment line_j at the j-th storage position that satisfies the set merging condition with line_i is determined, including the following steps:

[0103] S21. Extend at least one end of the currently to-be-processed lane line segment line_i in the first storage list by a first set length to obtain an extended line line_e at at least one end of line_i.

[0104] In the embodiment of the present application, the first reference points at the head and tail of the current lane line segment line_i in the first storage list can be selected according to the slope of line_i, and both ends of line_i are extended by a first set length based on the first reference points and the slope to obtain the extension lines line_e at both ends of line_i 1 and line_e 2 .

[0105] It should be noted that the first reference points in the embodiment of the present application do not refer to a single reference point, and the number thereof can be determined according to the calculation method for calculating the extension line, and the present application does not make any limitation thereto.

[0106] In addition, the first set length can include a horizontal length and a vertical length.

[0107] S22. Traverse the lane line segments in the first storage list from the (i + 1)-th storage position, and determine the lane line segments whose distances from line_e are less than a set threshold, and use them as the lane line segments to be screened.

[0108] In the embodiment of the present application, the remaining lane line segments in the first storage list can be traversed, and the shortest distance D 1 from the lane line segment to line_e 1 , and the shortest distance D 2 from the lane line segment to line_e 2 are calculated; the smaller value between D 1 and D 2 is compared with the set threshold; if the smaller value between D 1 and D 2 is less than the set threshold, then the lane line segment is used as the lane line segment to be screened.

[0109] In the embodiment of the present application, by separately determining whether the shortest distances from the lane line segment to the extension lines line_e 1 and line_e 2 at both ends of line_i are greater than the set threshold to determine the lane line segments to be screened, the method has higher accuracy compared with the method of only judging one end extension line and can prevent misprocessing.

[0110] S23. Extend at least one end of the lane line segment to be screened by a second set length to obtain the extension line line_e' at at least one end of the lane line segment to be screened.

[0111] In the embodiment of the present application, the second reference points at the head and tail of the lane line segment to be screened can be selected according to the slope of the lane line segment to be screened, and both ends of the lane line segment to be screened are extended by a second set length based on the second reference points and the slope to obtain the extension lines line_e 1 ' and line_e 2 '.

[0112] It should be noted that in the embodiments of the present application, the second reference point does not refer to a single reference point, and its quantity can be determined according to the calculation method of the calculated extension line. The present application does not make any limitation thereto.

[0113] In addition, the second set length may include a horizontal length and a vertical length.

[0114] S24. Calculate the included angle α between line_e' and line_e. If α is less than the set angle, then use the lane line segment to be screened as the lane line segment line_j.

[0115] The embodiments of the present application can calculate line_e 1 ’ and line_e 2 ’ respectively with line_e 1 ’s shortest distances D 1 ’ and D 1 ’’; and line_e 1 ’ and line_e 2 ’ respectively with line_e 2 ’s shortest distances D 2 ’ and D 2 ’’; select the extension line corresponding to the smaller value among D 1 ’ and D 1 ’’ as the first extension line to be screened; select the extension line corresponding to the smaller value among D 2 ’ and D 2 ’’ as the second extension line to be screened; calculate the included angle α 1 between the first extension line to be screened and line_e 1 , and the included angle α 2 between the second extension line to be screened and line_e 2 ; if α 1 and / or α 2 is less than the set angle, then use the lane line segment to be screened as the lane line segment line_j.

[0116] By calculating the distances and angles of two pairs of extension lines (line_e 1 ’ and line_e 2 ’, and line_e 1 and line_e 2 ), compared with only calculating the distances and angles of one pair of extension lines, the calculation accuracy is higher, and misprocessing can be reduced.

[0117] To further understand the definition of the present invention regarding "meeting the set merging conditions", the following will be elaborated in combination with specific embodiments. These embodiments are only used to illustrate the present application and not to limit the scope of the present application.

[0118] 1. Cluster the lane line pixel coordinates in each frame image, and fit lane line segments to each cluster in the clustering result to obtain lane line segment set data containing multiple lane line segment data. The fitting method can be least squares, B-spline, decision tree regression, etc., which is not limited in this application.

[0119] 2. Store multiple lane segment data in the lane segment set data in the first storage list list_lanes; then make a deep copy of the first storage list list_lanes to obtain a second storage list list_lanes_, and the data stored in the initial list_lanes_ is the same as that in list_lanes.

[0120] 3. Traverse the lane segment data stored in each storage position in list_lanes, and perform the following operations on the lane segment data of each storage position in list_lanes, such as the lane segment data list_lanes[i] of the i-th storage position:

[0121] 1) According to the first reference point A and the last reference point C of list_lanes[i], and the distance head_dis from the preset extension line reference point to the first reference point or the last reference point, obtain the first extension line reference point B and the last extension line reference point D on list_lanes[i], and mark A, B and C, D according to the direction of list_lanes[i]. Specifically: if A to B is east-west (the absolute value of the angle between list_lanes[i] and the north direction is greater than 45°, it is east-west), then mark A as P 1 , B is marked as P 2 ; If A to B is in the north-south direction (the absolute value of the angle between list_lanes[i] and the north direction is less than 45°, it is in the north-south direction), then mark A as P 2 , B is marked as P 1 ; If the direction from C to D is north-south, mark C as P 2 ', D is marked as P 1 ', and P 1 and P 2 The coordinates of are (x 1 _,y 1 _) and (x 2 _,y 2 _), P 2 ' and P 1 The coordinates of ' are (xx 2 _,yy 2 _), (xx 1 _,yy 1 _).

[0122] 2) According to [(x 1 _, y 1 _), (x 2 _, y 2 _)], [(xx 2 _, yy 2 _), (xx 1 _, yy 1 _)] and the preset horizontal length extend_dis[x] and vertical length extend_dis[y] of the extension line, generate the extension lines extend_shaPely_line and extend_shaPely_line1 at both ends of list_lanes[i]. Among them, the coordinates of the head or tail of the extension line are as follows:

[0123] (P 2 [0] + EXTRAPOL_RATIO[x] × (P 2 [0] - P 1 [0]), P 2 [1] + EXTRAPOL_RATIO[y] × (P 2 [1] - P 1 [1]))

[0124] Among them, P 2 [0] is the x coordinate of point P 2 or P 2 '; P 1 [0] is respectively the x coordinate of point P 1 or P 1 '; P 2 [1] is the y coordinate of point P 2 or P 2 '; P 1 [1] is respectively the y coordinate of point P 1 or P 1 '; EXTRAPOL_RATIO[x] = extend_dis[x] / dis[x], where dis[x] refers to the horizontal distance between two points P 1 and P 2 or the horizontal distance between two points P 1 ' and P 2 '; EXTRAPOL_RATIO[y] = extend_dis[y] / dis[y], where dis[y] refers to the vertical distance between two points P 1 and P 2 or the vertical distance between two points P 1 ' and P 2 '.

[0125] 4. For list_lanes[i], starting from the (i + 1)-th position, traverse the remaining list_lanes backward.

[0126] The following takes list_lanes[j] as an example to illustrate how to calculate whether list_lanes[i] and list_lanes[j] meet the merging conditions, where i and j respectively represent the positions of list_lanes[i] and list_lanes[j] in list_lanes.

[0127] 1) First, respectively obtain the shortest distance min_dis between list_lanes[j] and extend_shaPely_line, and the shortest distance min_dis1 between list_lanes[j] and extend_shaPely_line1. If the minimum value of min_dis and min_dis1 is less than the set threshold ePs, enter the following process; if not, it means that the merging conditions are not met and directly skip the following process.

[0128] 2) Similar to step 3, obtain the extended lines line_t and line_t1 at both ends of list_lanes[j].

[0129] 3) Calculate the shortest distances dis_0 and dis_00 between line_t and line_t1 and extend_shaPely_line respectively, and the shortest distances dis_1 and dis_11 between line_t and line_t1 and extend_shaPely_line1 respectively.

[0130] 4) Select one corresponding line from line_t and line_t1 as line_tmP according to the minimum value of dis_0 and dis_00, and select one corresponding line from line_t and line_t1 as line_tmP1 according to the minimum value of dis_1 and dis_11.

[0131] 5) Calculate the angle angle between line_tmP and extend_shaPely_line respectively, and the angle angle1 between line_tmP_1 and extend_shaPely_line_1. In the embodiments of the present application, for the convenience of calculation, the angle is converted to 0~90°. The following takes angle as an example to illustrate the way of angle conversion:

[0132] If angle > 90° and angle < 180°, then convert angle to 180° - angle;

[0133] If angle ≥ 90° and angle < 270°, then convert angle to angle ~ 180°;

[0134] If angle ≥ 270° and angle ≤ 360°, then convert angle to 360° ~ angle.

[0135] 6) If either angle or angle1 is less than the set threshold, it means that list_lanes[j] can be merged with list_lanes[i].

[0136] Furthermore, in the embodiment of the present application, taking the case where there are five short lines line_1, line_2, line_3, line_4, line_5 stored in list_lanes, and line_1, line_3, line_5 need to be merged into one line as an example, the method for processing lane line segments in the embodiment of the present application is described:

[0137] Initially, list_lanes and list_lanes_ are the same, see Figure 4 and Figure 5(a);

[0138] Take line_1 in list_lanes as the currently to-be-processed lane line segment, traverse the remaining lane line segments, and according to the above steps, determine that line_3 and line_1 meet the preset merging conditions, then mark the position of line_3 in list_lanes_ as 1, and link line_3 to the position of line_1, see Figure 5(b);

[0139] Take line_3 in list_lanes as the currently to-be-processed lane line segment, traverse the remaining lane line segments, and according to the above steps, determine that line_5 and line_3 meet the preset merging conditions, then mark the position of line_5 in list_lanes_ as 3, and according to the index value 1 of the position where line_3 is located, link line_5 to the position of line_1, see Figure 5(c).

[0140] According to list_lanes_, it can be determined that line_1, line_3, and line_5 can be merged into one line, completing one cycle. And since line_3 and line_5 are both linked to the position where line_1 is located, then the values of the positions where line_3 and line_5 are located are set values, for example, 1. During the subsequent next cycle processing, these positions with set value 1 can be filtered out, and the cycle can be restarted from the positions with set value not 1. For example, the next cycle processing can start from line_2 in list_lanes.

[0141] Corresponding to the foregoing embodiments of the application function implementation method, the present application further provides a lane line processing device, an electronic device, and corresponding embodiments.

[0142] Figure 6 It is a schematic structural diagram of the lane line processing device shown in the embodiments of the present application.

[0143] See Figure 6 , the embodiments of the present application further provide a lane line processing device, including a generation module 610, a calculation module 620, a marking module 630, and an output module 640.

[0144] The generation module 610 is used to generate a first storage list and a second storage list for storing a lane line segment data set. The number of storage bits in the first storage list and the second storage list is not less than the total amount of lane line segments, and one storage bit stores one lane line segment data. Preferably, the number of storage bits in the first storage list and the second storage list is the same as the total amount of lane line segments; and / or, the same storage bit in the first storage list and the second storage list stores the same lane line segment.

[0145] The calculation module 620 is used to use the lane line segment line_i in the i-th storage bit of the first storage list as the lane line segment to be processed, traverse the lane line segments in the first storage list from the (i + 1)-th storage bit until the lane line segment line_j in the j-th storage bit that satisfies the set merging condition with line_i is determined, where j > i, and i = 1, 2, 3,..., n - 1, and n represents the total number of storage bits storing lane line segment data in the first storage list. Specifically, the calculation module 620 is used to: extend at least one end of the current lane line segment line_i in the first storage list by a first set length to obtain an extension line line_e of at least one end of line_i; traverse the lane line segments in the first storage list from the (i + 1)-th storage bit, determine the lane line segments whose distance from line_e is less than the set threshold, and use them as the lane line segments to be screened; extend at least one end of the lane line segments to be screened by a second set length to obtain an extension line line_e' of at least one end of the lane line segments to be screened; calculate the included angle α between line_e' and line_e, and if α is less than the set angle, use the lane line segments to be screened as the lane line segment line_j.

[0146] The marking module 630 is used to mark the index value of the storage bit where the lane line segment line_j' corresponding to line_j in the second storage list is located as i', and link line_j' to the storage bit corresponding to the index value i' according to the index value i' of the storage bit where line_j' is located, where i' represents the storage bit where the lane line segment line_i' corresponding to line_i in the second storage list is located; the calculation module is further used to use the lane line segment line_i+1 in the (i + 1)-th storage bit in the first storage list as the next lane line segment to be processed, and repeat traversing and calculating until the linking of the lane line segments in the second storage list and the marking of the index values of their storage bits are completed. The calculation module is further used to use the lane line segment line_i+1 in the (i + 1)-th storage bit in the first storage list as the next lane line segment to be processed, and repeat traversing and calculating until the linking of the lane line segments in the second storage list and the marking of the index values of their storage bits are completed. Preferably, the marking module 630 is used to directly link line_j' to the storage bit corresponding to the index value i' if the storage bit where the lane line segment line_i' corresponding to line_i in the second storage list is located is not marked with an index value; if the storage bit where the lane line segment line_i' corresponding to line_i in the second storage list is marked with an index value, link line_j' to the storage bit corresponding to the index value of the storage bit where line_i' is located.

[0147] The output module 640 is used to output the target lane line data according to the lane line segment data set in the second storage list that has been completed with linking and index value marking. Preferably, the output module 640 is used to filter the storage bits marked with index values in the second storage list, and use the lane line segment data set in the remaining storage bits as the target lane line data.

[0148] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0149] Figure 7 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.

[0150] See Figure 7 , the electronic device 700 includes a memory 710 and a processor 720.

[0151] The processor 720 may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0152] The memory 710 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, the ROM may store static data or instructions required by the processor 720 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (such as a magnetic or optical disk, flash memory) as the permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. The system memory may store some or all of the instructions and data required by the processor during operation. In addition, the memory 710 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used. In some embodiments, the memory 710 may include a removable storage device that is readable and / or writable, such as a compact disc (CD), read-only digital versatile disc (such as DVD-ROM, dual-layer DVD-ROM), read-only Blu-ray disc, super density disc, flash memory card (such as SD card, min SD card, Micro-SD card, etc.), magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and instantaneous electronic signals transmitted wirelessly or by wire.

[0153] An executable code is stored on the memory 710, and when the executable code is processed by the processor 720, it may cause the processor 720 to execute some or all of the methods described above.

[0154] In addition, the method according to the present application can also be implemented as a computer program or a computer program product, which includes computer program code instructions for performing some or all of the steps in the above method of the present application.

[0155] Alternatively, the present application can also be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium), on which executable code (or a computer program or computer instruction code) is stored. When the executable code (or the computer program or computer instruction code) is executed by a processor of an electronic device (or a server, etc.), the processor is caused to execute some or all of the steps of the above method according to the present application.

[0156] The embodiments of the present application have been described above. The above description is exemplary and not exhaustive, and is also not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the embodiments disclosed herein.

Claims

1. A lane line processing method, characterized in that: The following steps are involved: S1. Generate a first storage list and a second storage list for storing lane segment data sets, wherein one storage bit in the first storage list and the second storage list stores one lane segment data, and the distribution of the lane segment data in the first storage list and the second storage list is the same; S2. Take the lane segment line_i in the i-th storage position in the first storage list as the lane segment to be processed, and traverse the lane segments in the first storage list from the i+1-th storage position until the lane segment line_j in the j-th storage position that meets the set merging condition with line_i is determined, where j>i, and i=1, 2, 3, ..., n-1, and n represents the total number of storage positions in the first storage list; S3, marking the index value of the storage position where the lane segment line_j' corresponding to line_j in the second storage list is located as i', and linking line_j' to the storage position corresponding to the index value i', wherein i' represents the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located; S4, taking the lane segment line_i+1 in the i+1th storage position in the first storage list as the next lane segment to be processed, and repeating S2 and S3 until the links of the lane segments in the second storage list and the index value markings of their storage positions are completed; S5. Output target lane line data according to the lane line segment data set in the second storage list that has been linked and marked with index values.

2. The method according to claim 1, characterized in that The number of storage bits in the first storage list and the second storage list is not less than the total number of lane segments.

3. The method according to claim 2, characterized in that The same storage position in the first storage list and the second storage list stores a same lane line segment.

4. The method according to claim 1, characterized in that: Outputting target lane line data according to the lane line segment data set in the second storage list that has completed linking and index value marking comprises the following steps: The storage bits marked with index values ​​in the second storage list are filtered and deleted, and the lane line segment data sets in the retained storage bits are used as target lane line data.

5. The method according to claim 1, characterized in that The linking of line_j' to the storage bit corresponding to the index value i' comprises the following steps: If the storage position where the lane segment line_i' corresponding to line_i is located in the second storage list is not marked with an index value, directly link line_j' to the storage position corresponding to the index value i'; If the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located is marked with an index value, line_j' is linked to the storage position corresponding to the index value of the storage position where line_i' is located.

6. The method according to claim 1, characterized in that The method of taking the lane segment line_i in the i-th storage position in the first storage list as the lane segment to be processed, and traversing the lane segments in the first storage list from the i+1-th storage position until determining the lane segment line_j in the j-th storage position that satisfies the set merging condition with line_i includes the following steps: Extend at least one end of the current lane line segment line_i to be processed in the first storage list by a first set length to obtain an extension line line_e of at least one end of line_i; Traverse the lane line segments in the first storage list from the i+1th storage position, determine the lane line segment whose distance to line_e is less than the set threshold, and use it as the lane line segment to be screened; Extend at least one end of the lane segment to be screened by a second set length to obtain an extension line line_e' of at least one end of the lane segment to be screened; The angle α between line_e' and line_e is calculated. If α is less than the set angle, the lane segment to be screened is used as the lane segment line_j.

7. The method according to claim 6, characterized in that: The step of extending at least one end of the current lane line segment line_i to be processed in the first storage list by a first set length to obtain an extension line line_e of at least one end of line_i includes the following steps: According to the slope of the current lane segment line_i to be processed in the first storage list, select the first reference points at the beginning and the end of line_i, and extend the first set length at both ends of line_i based on the first reference points to obtain the extended lines line_e1 and line_e2 at both ends of line_i; and / or, The step of extending at least one end of the lane segment to be screened by a second set length to obtain an extension line line_e' of at least one end of the lane segment to be screened comprises the following steps: According to the slope of the lane segment to be screened, second reference points at the beginning and end of the lane segment to be screened are selected, and based on the second reference points, both ends of the lane segment to be screened are extended by a second set length to obtain extended lines line_e1' and line_e2' at both ends of the lane segment to be screened.

8. The method according to claim 7, characterized in that The step of traversing the lane segments in the first storage list from the i+1th storage position to determine the lane segments whose distance to line_e is less than a set threshold and using them as lane segments to be screened includes the following steps: Traverse the lane segments in the first storage list from the i+1th storage position, and calculate the shortest distance D1 between each lane segment and line_e1, and the shortest distance D2 between each lane segment and line_e2 in turn; Compare the smaller value of D1 and D2 with the set threshold; If the smaller value of D1 and D2 is less than the set threshold, the lane segment is used as the lane segment to be screened.

9. The method according to claim 7, characterized in that: Before calculating the angle α between line_e' and line_e, the following steps are included: Calculate the shortest distances D1' and D1'' between line_e1' and line_e2' and line_e1, and the shortest distances D2' and D2'' between line_e1' and line_e2' and line_e2; Select the extension line corresponding to the smaller value of D1' and D1'' and use it as the first extension line to be screened; The extension line corresponding to the smaller value of D2' and D2'' is selected as the second extension line to be screened.

10. The method according to claim 9, characterized in that The calculating of the angle α between line_e' and line_e, and if α is less than a set angle, taking the lane segment to be screened as lane segment line_j, comprises the following steps: Calculate the angle α1 between the first extended line to be screened and line_e1, and the angle α2 between the second extended line to be screened and line_e2; If α1 and / or α2 is smaller than the set angle, the lane line segment to be screened is used as lane line segment line_j.

11. A lane line processing device, characterized in that: include: A generating module, configured to generate a first storage list and a second storage list for storing lane segment data sets, wherein the number of storage bits in the first storage list and the second storage list is not less than the total number of lane segments, and one storage bit stores one lane segment data; and the distribution of the lane segment data in the first storage list and the second storage list is the same; a calculation module, configured to use the lane segment line_i in the i-th storage position in the first storage list as the lane segment to be processed, and traverse the lane segments in the first storage list from the i+1-th storage position until determining the lane segment line_j in the j-th storage position that satisfies the set merging condition with line_i, wherein j>i, and i=1, 2, 3, ..., n-1, and n represents the total number of storage positions in the first storage list storing lane segment data; a marking module, used to mark the index value of the storage position where the lane segment line_j' corresponding to line_j in the second storage list is located as i', and link line_j' to the storage position corresponding to the index value i', wherein i' represents the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located; the calculation module is also used to take the lane segment line_i+1 in the i+1th storage position in the first storage list as the next lane segment to be processed, and repeat traversal and calculation until the linking of the lane segments in the second storage list and the index value marking of their storage positions are completed; An output module is used to output target lane line data according to the lane line segment data set in the second storage list that has completed linking and index value marking.

12. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method as claimed in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: An executable code is stored thereon, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as claimed in any one of claims 1 to 10.

Images