Lane line processing method and device, equipment and storage medium

Abstract

The invention relates to a lane line processing method and device, equipment and a storage medium. The method comprises the following steps: generating a first storage list and a second storage list for storing a lane line segment data set; taking the lane line segment linei in the first storage list as a to-be-processed lane line segment, and traversing the lane line segment in the first storage list from the (i + 1) th storage bit until a lane line segment linej meeting a set merging condition is determined; marking an index value of a storage bit where a lane line segment (linej ') corresponding to the linej in the second storage list is located as i', and linking the linej 'to the storage bit corresponding to the index value i'; taking the lane line segment linei + 1 as the next to-be-processed lane line segment until the link of the lane line segments in the second storage list and the index value marking of the storage bits of the lane line segments are completed; and outputting target lane line data according to the lane line segment data set in the second storage list which is linked and marked by the index value. According to the scheme provided by the invention, the problems of wrong merging and missing merging can be avoided, and the accuracy of lane line merging is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a lane line processing method, device, equipment and storage medium in electronic map production. Background Art

[0002] Lane lines are the left and right side lines that determine the range of a lane. Generally, the lane range is determined by lane lines printed on the ground. They are roughly divided into five types: single dotted line, single solid line, double solid line, double dotted line, and dotted and solid lines. They ensure that vehicles are driving in the correct lane and provide safety for vehicle driving. They are the most important road elements in the production of high-precision maps for autonomous driving.

[0003] In the related art, a lane line image is obtained, and instance segmentation is used to determine whether the pixel data in the same image belongs to the same lane line. However, this method cannot determine whether lane line segments in different images belong to the same lane line.

[0004] To solve the above problem, the distance from the pixel coordinates of the lane line to the vehicle is generally used to confirm whether the lane line segments in different images belong to the same lane line. However, if the vehicle changes lanes during driving, or there is a situation where lane lines merge or diverge, this method cannot make accurate distinctions. Summary of the invention

[0005] In order to solve or partially solve the problems existing in the related art, the present application provides a lane line processing method, device, equipment and storage medium, which can avoid the problems of mismerging and missed merging in lane line processing and improve the accuracy of lane line merging.

[0006] The first aspect of the present application provides a lane line processing method, comprising the following steps: S1, generating a first storage list and a second storage list for storing lane segment data sets, wherein one storage bit in the first storage list and one storage bit in the second storage list stores one lane segment data; S2. Take the lane segment line_i in the i-th storage position in the first storage list as the lane segment to be processed, and traverse the lane segments in the first storage list from the i+1-th storage position until the lane segment line_j in the j-th storage position that meets the set merging condition with line_i is determined, where j>i, and i=1, 2, 3, ..., n-1, and n represents the total number of storage positions in the first storage list; S3, marking the index value of the storage position where the lane segment line_j' corresponding to line_j in the second storage list is located as i', and linking line_j' to the storage position corresponding to the index value i', wherein i' represents the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located; S4, taking the lane segment line_i+1 in the i+1th storage position in the first storage list as the next lane segment to be processed, and repeating S2 and S3 until the links of the lane segments in the second storage list and the index value markings of their storage positions are completed; S5. Output target lane line data according to the lane line segment data set in the second storage list that has been linked and marked with index values.

[0007] As an optional embodiment, the number of storage bits in the first storage list and the second storage list is not less than the total number of lane segments.

[0008] As an optional embodiment, the same storage position in the first storage list and the second storage list stores a same lane line segment.

[0009] As an optional embodiment, the outputting of the target lane line data according to the lane line segment data set in the second storage list that has completed linking and index value marking comprises the following steps: The storage bits marked with index values ​​in the second storage list are filtered and deleted, and the lane line segment data sets in the retained storage bits are used as target lane line data.

[0010] As an optional embodiment, linking line_j' to the storage bit corresponding to the index value i' comprises the following steps: If the storage position where the lane segment line_i' corresponding to line_i is located in the second storage list is not marked with an index value, directly link line_j' to the storage position corresponding to the index value i'; If the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located is marked with an index value, line_j' is linked to the storage position corresponding to the index value of the storage position where line_i' is located.

[0011] As an optional embodiment, the lane segment line_i in the i-th storage position in the first storage list is used as the lane segment to be processed, and the lane segments in the first storage list are traversed from the i+1-th storage position until the lane segment line_j in the j-th storage position that satisfies the set merging condition with line_i is determined, including the following steps: Extend at least one end of the current lane line segment line_i to be processed in the first storage list by a first set length to obtain an extension line line_e of at least one end of line_i; Traverse the lane line segments in the first storage list from the i+1th storage position, determine the lane line segment whose distance to line_e is less than the set threshold, and use it as the lane line segment to be screened; Extend at least one end of the lane segment to be screened by a second set length to obtain an extension line line_e' of at least one end of the lane segment to be screened; The angle α between line_e' and line_e is calculated. If α is less than the set angle, the lane segment to be screened is used as the lane segment line_j.

[0012] As an optional embodiment, the step of extending at least one end of the current lane line segment line_i to be processed in the first storage list by a first set length to obtain an extension line line_e of at least one end of line_i includes the following steps: According to the slope of the current lane segment line_i to be processed in the first storage list, the first reference points at the beginning and end of line_i are selected, and based on the first reference points, both ends of line_i are extended by a first set length to obtain extended lines line_e1 and line_e2 at both ends of line_i.

[0013] As an optional embodiment, the step of extending at least one end of the lane segment to be screened by a second set length to obtain an extension line line_e' of at least one end of the lane segment to be screened includes the following steps: According to the slope of the lane segment to be screened, second reference points at the beginning and end of the lane segment to be screened are selected, and based on the second reference points, both ends of the lane segment to be screened are extended by a second set length to obtain extended lines line_e1' and line_e2' at both ends of the lane segment to be screened.

[0014] As an optional embodiment, traversing the lane segments in the first storage list from the i+1th storage bit, determining the lane segments whose distance to line_e is less than a set threshold and using them as lane segments to be screened, includes the following steps: Traverse the lane segments in the first storage list from the i+1th storage position, and calculate the shortest distance D1 between each lane segment and line_e1, and the shortest distance D2 between each lane segment and line_e2 in turn; Compare the smaller value of D1 and D2 with the set threshold; If the smaller value of D1 and D2 is less than the set threshold, the lane segment is used as the lane segment to be screened.

[0015] As an optional embodiment, before calculating the angle α between line_e' and line_e, the following steps are included: Calculate the shortest distances D1' and D1'' between line_e1' and line_e2' and line_e1, and the shortest distances D2' and D2'' between line_e1' and line_e2' and line_e2, respectively; Select the extension line corresponding to the smaller value of D1' and D1'' and use it as the first extension line to be screened; The extension line corresponding to the smaller value of D2' and D2'' is selected as the second extension line to be screened.

[0016] As an optional embodiment, the calculating of the angle α between line_e' and line_e, if α is less than a set angle, taking the lane segment to be screened as lane segment line_j, comprises the following steps: Calculate the angle α1 between the first extended line to be screened and line_e1, and the angle α2 between the second extended line to be screened and line_e2; If α1 and / or α2 is smaller than the set angle, the lane line segment to be screened is used as lane line segment line_j.

[0017] A second aspect of the present application provides a lane line processing device, comprising: A generating module, used to generate a first storage list and a second storage list for storing lane segment data sets, wherein the number of storage bits in the first storage list and the second storage list is not less than the total number of lane segments, and one storage bit stores one lane segment data; a calculation module, configured to use the lane segment line_i in the i-th storage position in the first storage list as the lane segment to be processed, and traverse the lane segments in the first storage list from the i+1-th storage position until determining the lane segment line_j in the j-th storage position that satisfies the set merging condition with line_i, wherein j>i, and i=1, 2, 3, ..., n-1, and n represents the total number of storage positions in the first storage list storing lane segment data; a marking module, used to mark the index value of the storage position where the lane segment line_j' corresponding to line_j in the second storage list is located as i', and link line_j' to the storage position corresponding to the index value i', wherein i' represents the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located; the calculation module is also used to take the lane segment line_i+1 in the i+1th storage position in the first storage list as the next lane segment to be processed, and repeat traversal and calculation until the linking of the lane segments in the second storage list and the index value marking of their storage positions are completed; An output module is used to output target lane line data according to the lane line segment data set in the second storage list that has completed linking and index value marking.

[0018] A third aspect of the present application provides an electronic device, including: Processor; and The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0019] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0020] The technical solution provided by this application may have the following beneficial effects: The present application generates a first storage list for storing a set of lane segments, and a second storage list matching the first storage list. In the process of processing lane lines, the merging conditions of the lane segment data in the first storage list are judged. The application can also mark index values ​​of the mergeable lane segment data in the second storage list by comparing the first storage list without changing the lane segment data stored in the first storage list. The application can then accurately locate the location of the mergeable lane segment data according to the index value, thereby achieving rapid merging processing of the corresponding lane segments and avoiding incorrect merging and missed merging of lane segments.

[0021] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0023] Figure 1is a flowchart of a lane line processing method shown in an embodiment of the present application; Figure 2 is a schematic diagram of the structure of the first storage list shown in an embodiment of the present application; FIG3 (a) is a schematic diagram of an initial pattern structure of a second storage list shown in an embodiment of the present application; FIG3( b ) is a schematic diagram of the structure of the second storage list completion mark shown in FIG3( a ); FIG3( c ) is a schematic diagram of the structure of the second storage list shown in FIG3( b ) after merging; Figure 4 is another structural schematic diagram of the first storage list shown in an embodiment of the present application; FIG5( a ) is another structural diagram of an initial format of a second storage list shown in an embodiment of the present application; FIG5( b ) is a schematic diagram of the structure of the second storage list completion mark shown in FIG5( a ); FIG5(c) is a schematic diagram of the structure of the second storage list shown in FIG5(b) after merging; Figure 6 is a schematic diagram of the structure of a lane line processing device shown in an embodiment of the present application; Figure 7 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0025] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0027] In the related art, lane line images are obtained and instance segmentation is used to determine whether the pixel data in the same image belongs to the same lane line. However, this method cannot determine whether lane line segments in different images belong to the same lane line. To solve the above problem, the distance from the pixel coordinates of the lane line to the vehicle is generally used to determine whether the lane line segments in different images belong to the same lane line. However, if the vehicle changes lanes during driving, or there is a situation where the lane lines merge or diverge, this method cannot accurately distinguish them.

[0028] In response to the above problems, an embodiment of the present application provides a lane line processing method, which can avoid the problems of mis-merging and missed merging in lane line processing and improve the accuracy of lane line merging.

[0029] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0030] Figure 1 It is a flowchart of the lane line processing method shown in an embodiment of the present application.

[0031] See also Figure 1 , the embodiment of the present application provides a lane line processing method, including the following steps S1 to S5: S1. Generate a first storage list and a second storage list for storing lane segment data sets, wherein one storage bit in the first storage list and the second storage list stores one lane segment data.

[0032] The lane line segment data set in the embodiment of the present application may refer to a plurality of lane line segment data obtained by distance clustering of lane line pixel coordinate points in a plurality of frames of images, wherein one lane line segment data is obtained by clustering each frame of images.

[0033] In the embodiment of the present application, the plurality of lane line segment data obtained above can be stored in a plurality of storage bits of the first storage list and the second storage list, wherein the format of the first storage list can be seen in Figure 2 .

[0034] The distribution of lane segment data in the first storage list and the second storage list is the same, for example, it can be concentrated or discontinuous. The embodiment of the present application preferably has a concentrated distribution.

[0035] Specifically, the same storage positions in the first storage list and the second storage list store the same lane segment data, or the relative position relationship of the storage positions where the lane segment data in the first storage list are located is the same as those in the second storage list, so that the storage positions where the lane segment data in the first storage list are located and the storage positions where the lane segment data in the second storage list are located can be mapped to each other.

[0036] In the embodiment of the present application, the initial style (which may refer to the data distribution method) of the second storage list may be completely the same as that of the first storage list, wherein the style of the first storage list can be referred to in Figure 2 The initial format of the second storage list can be seen in Figure 3 (a). During the lane line processing, the data in the first storage list can remain unchanged, and the data in the first storage list is mainly used for the calculation of the merging condition. Its format is always as follows: Figure 2 . The data in the second storage list will change with the calculation results of the first storage list. For example, the second storage list can be marked with index values ​​according to the calculation results of the first storage list, and the lane line segment data can be linked to the corresponding storage bit according to the index value (the "link" here can be understood as moving the lane line segment data to (or merging it to) the storage bit corresponding to its index value). Therefore, the lane line data in the storage bit of the second storage list is always changing. The embodiment of the present application can not only keep the lane line segment data used for merging calculation unchanged through the cooperation of the two storage lists, but also mark and merge the lane line segment data that meets the merging conditions, which can greatly reduce the amount of data processing and improve processing efficiency.

[0037] Preferably, the number of storage bits in the first storage list and the second storage list is not less than the total number of lane line segments. For example, n lane line data are placed in the first storage list and the second storage list having at least n storage bits, and the first storage bit storing lane line segment data is used as the first storage bit, the i-th storage bit storing lane line segment data is used as the i-th storage bit, the j-th storage bit storing lane line segment data is used as the j-th storage bit, and the n-th storage bit storing lane line segment data is used as the n-th storage bit.

[0038] Further preferably, the number of storage bits in the first storage list and the second storage list is the same as the total number of lane segments.

[0039] In the embodiment of the present application, the number of storage bits of the first storage list and the second storage list can be set according to the total amount of actual lane segment data, so that each storage bit of the first storage list and the second storage list stores one lane segment data, which can further reduce the amount of calculation.

[0040] Preferably, the same storage location in the first storage list and the second storage list stores a same lane line segment.

[0041] The embodiment of the present application can also copy the first storage list to obtain the second storage list, so that each identical storage position in the first storage list and the second storage list stores an identical lane line segment, which can avoid repeated storage of a large amount of lane line data twice.

[0042] S2. Take the lane segment line_i in the i-th storage position in the first storage list as the lane segment to be processed, and traverse the lane segments in the first storage list from the i+1-th storage position until the lane segment line_j in the j-th storage position that meets the set merging condition with line_i is determined, where j>i, and i=1, 2, 3, ..., n-1, and n represents the total number of storage positions in the first storage list that store lane segment data.

[0043] The initial lane line segment to be processed in the embodiment of the present application can be arbitrarily selected from the first storage list, and is preferably the lane line segment line_1 in the first storage position.

[0044] Based on the lane segment line_i in the i-th storage position, when traversing other lane segments in the first storage list, the embodiment of the present application starts from the i+1-th storage position and traverses from the back, rather than traversing from the first position to the back, and only traverses the storage positions greater than i, which can reduce the number of traversals and improve the efficiency of lane line processing.

[0045] After the embodiment of the present application determines the lane segment line_j that meets the set merging condition with line_i, the lane segment in the i+1th storage position is used as the next lane segment to be processed, so as to avoid missing lane segment data. And starting from i to n-1, that is, the last storage position (nth storage position) storing lane segment data is not merged. This is because there are two possibilities for the lane segment data in the nth storage position, one is that it already has an index value, and the other is that the lane segment data from the i-th storage position to the n-th storage position do not meet the merging condition, so there is no need to merge the n-th storage position, which can further save the amount of calculation.

[0046] In an embodiment of the present application, the following method can be used to determine the lane segment line_j in the jth storage position that meets the set merging condition with the lane segment line_i in the ith storage position: extend at least one end of the current lane segment line_i to be processed in the first storage list by a first set length to obtain an extension line line_e of at least one end of line_i; traverse the lane segments in the first storage list from the i+1th storage position, determine the lane segment whose distance to line_e is less than a set threshold, and use it as the lane segment to be screened; extend at least one end of the lane segment to be screened by a second set length to obtain an extension line line_e' of at least one end of the lane segment to be screened; calculate the angle α between line_e' and line_e, and if α is less than the set angle, use the lane segment to be screened as the lane segment line_j.

[0047] The embodiment of the present application determines whether the set merging conditions are met from multiple aspects such as the distance and angle of the extension lines of two lane segments, which can reduce the erroneous merging and missed merging of lane segments.

[0048] S3. Mark the index value of the storage position where the lane segment line_j' corresponding to line_j in the second storage list is located as i', and link line_j' to the storage position corresponding to the index value i', wherein i' represents the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located.

[0049] In the embodiment of the present application, the distribution of the storage bits storing lane segment data in the first storage list can be compared to determine the storage bit where the lane segment line_j' corresponding to the lane segment line_j in the j-th storage bit in the first storage list and the storage bit where the lane segment line_i' corresponding to the lane segment line_i in the ith storage bit in the first storage list are located from the second storage list. This is because the lane segment data in the two storage lists are distributed in the same manner or mapped in the storage bits, so the data of the lane segment line_j in the j-th storage bit in the first storage list is the same as the data of the lane segment line_j' in the j'th storage bit in the second storage list, and the data of the lane segment line_i in the ith storage bit in the first storage list is the same as the data of the lane segment line_i' in the ith storage bit in the second storage list. Therefore, the storage bit in the second storage list storing the same lane segment data can be determined based on the position of line_i and line_j in the first storage list. Here, i and i' as well as j and j' may be identical or mapped in value, for example, i and i' are both 1 or both refer to the first storage bit, and j and j' are both 3 or both refer to the third storage bit.

[0050] Specifically, referring to FIG. 3 (b), the embodiment of the present application marks the index value of the storage position where the lane segment line_j' corresponding to line_j in the second storage list is located as i'. According to the index value i' marked at the storage position where line_j' is located, it can be determined that the storage position where the lane segment that meets the set merging condition with line_j' is located is the i'th storage position, and the lane segment stored in the i'th storage position is line_i'; referring to FIG. 3 (c), the lane segment line_j' is then linked to the i'th storage position to realize the data merging of line_j' and line_i'.

[0051] Preferably, if the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located is not marked with an index value, it means that the lane segment before the storage position of lane segment line_i' does not meet the set merging condition with the lane segment line_i', or it means that the lane segment before the storage position of lane segment line_i' has not been calculated with the lane segment line_i', then line_j' can be directly linked to the storage position corresponding to the index value i'.

[0052] If the storage position of the lane segment line_i' corresponding to line_i in the second storage list is marked with an index value, it means that the lane segment before the storage position of lane segment line_i' meets the set merging condition with the lane segment line_i', and the lane segment line_i' has been linked to the storage position corresponding to its index value, then the lane segment line_j' is also linked to the storage position corresponding to the index value of the storage position where line_i' is located.

[0053] S4. Use the lane segment line_i+1 in the i+1th storage position in the first storage list as the next lane segment to be processed, and repeat S2 and S3 until the linking of the lane segments in the second storage list and the index value marking of their storage positions are completed.

[0054] The lane segment line_i+1 in the i+1th storage position in the first storage list is taken as the next lane segment to be processed, and the above steps S2 and S3 are repeated. The merge calculation can be completed from the i-th storage position to the n-1-th storage position in the first storage list, thereby improving the comprehensiveness of the calculation.

[0055] S5. Generate target lane line data according to the lane line segment data set in the second storage list that has been linked and marked with index values.

[0056] Based on the above steps, after completing the linking of the lane segments in the second storage list and marking of their index values, the storage bits corresponding to the mergeable lane segments in the second storage list are marked with index values, and the lane segments marked with index values ​​have been linked to the storage bits corresponding to the index values, and the merged lane segment data can be obtained based on the linking and indexing results.

[0057] The embodiment of the present application generates a first storage list and a second storage list for storing a lane segment set. During the process of processing lane lines, the lane segment data in the first storage list remains unchanged, the merging conditions are calculated for the lane segment data in the first storage list, and then the index values ​​of the mergeable lane segment data in the second storage list are marked by comparing with the first storage list. According to the index value, the lane segment stored in the rear position is accurately linked to the storage position where the lane segment data stored in the front position and can be merged is located, thereby realizing fast merging processing of the corresponding lane segments and avoiding incorrect merging and missed merging of lane segments.

[0058] As an optional embodiment, step S5 outputs the target lane line data according to the lane line segment data set in the second storage list that has completed linking and index value marking, including the following steps: The storage bits marked with index values ​​in the second storage list are filtered and deleted, and the lane line segment data set in the retained storage bits is used as the target lane line data and output.

[0059] The lane line segment data in the storage bit marked with the completed index value in the second storage list of the embodiment of the present application has been linked to the corresponding storage bit, so the numerical value in the storage bit marked with the completed index value is a set value, such as 1. When outputting the target lane line data, the position where the storage bit is the set value is filtered and deleted, and the remaining storage bit contains several lane line segment data merged together, see Figure 3 (c), in which the values ​​of the storage bits where the lane line segments with marked index values ​​are located are all set to the set values.

[0060] Since the marked lane segments are all linked to the positions of the lane segments corresponding to the index values, the values ​​of the positions of the marked lane segments are all set values, such as 1. In the next subsequent loop processing, these positions with set values ​​of 1 can be filtered out, and the loop can be started again from the positions with set values ​​not 1 to avoid redundant calculations.

[0061] As an optional embodiment, in step S2, the current lane segment line_i to be processed in the first storage list is determined, and the lane segments in the first storage list are traversed from the i+1th storage position until the lane segment line_j in the jth storage position that satisfies the set merging condition with line_i is determined, including the following steps: S21. Extend at least one end of the current lane line segment line_i to be processed in the first storage list by a first set length to obtain an extension line line_e of at least one end of line_i.

[0062] In the embodiment of the present application, the first reference points at the beginning and end of line_i can be selected according to the slope of the current lane segment line_i to be processed in the first storage list, and the first set length is extended at both ends of line_i based on the first reference point and the slope to obtain the extended lines line_e1 and line_e2 at both ends of line_i.

[0063] It should be noted that the first reference point in the embodiment of the present application does not refer to one reference point, and its number can be determined according to the calculation method of the extension line, and the present application does not limit this.

[0064] In addition, the first set length may include a horizontal length and a vertical length.

[0065] S22. Traverse the lane segments in the first storage list from the i+1th storage position, determine the lane segments whose distance to line_e is less than a set threshold, and use them as the lane segments to be screened.

[0066] The embodiment of the present application can traverse the remaining lane segments in the first storage list, and calculate the shortest distance D1 between the lane segment and line_e1, and the shortest distance D2 between the lane segment and line_e2; compare the smaller value of D1 and D2 with the set threshold; if the smaller value of D1 and D2 is less than the set threshold, the lane segment is used as the lane segment to be screened.

[0067] The embodiment of the present application determines the lane segment to be screened by respectively judging whether the shortest distance between the lane segment and the extension lines line_e1 and line_e2 at both ends of line_i is greater than a set threshold. This method is more accurate than the method of only judging the extension line at one end and can prevent misprocessing.

[0068] S23, extending at least one end of the lane segment to be screened by a second set length to obtain an extension line line_e' of at least one end of the lane segment to be screened.

[0069] In the embodiment of the present application, the second reference points at the beginning and end of the lane segment to be screened can be selected according to the slope of the lane segment to be screened, and the second set length can be extended at both ends of the lane segment to be screened based on the second reference point and the slope to obtain the extended lines line_e1' and line_e2' at both ends of the lane segment to be screened.

[0070] It should be noted that the second reference point in the embodiment of the present application does not refer to one reference point, and its number can be determined according to the calculation method of the extension line, and the present application does not limit this.

[0071] In addition, the second set length may include a horizontal length and a vertical length.

[0072] S24. Calculate the angle α between line_e' and line_e. If α is less than the set angle, the lane segment to be screened is used as the lane segment line_j.

[0073] The embodiment of the present application can calculate the shortest distances D1' and D1'' between line_e1' and line_e2' and line_e1 respectively, and the shortest distances D2' and D2'' between line_e1' and line_e2' and line_e2 respectively; select the extension line corresponding to the smaller value of D1' and D1'', and use it as the first extension line to be screened; select the extension line corresponding to the smaller value of D2' and D2'', and use it as the second extension line to be screened; calculate the angle α1 between the first extension line to be screened and line_e1, and the angle α2 between the second extension line to be screened and line_e2; if α1 and / or α2 are less than the set angle, the lane line segment to be screened is used as the lane line segment line_j.

[0074] In the embodiment of the present application, the distance and angle of two pairs of extension lines (line_e1' and line_e2', and line_e1 and line_e2) are calculated. Compared with calculating the distance and angle of only one pair of extension lines, the calculation accuracy is higher and erroneous processing can be reduced.

[0075] In order to further understand the definition of "satisfying the set merging conditions" in the present invention, it is explained below in conjunction with specific embodiments. These embodiments are only used to illustrate the present application and are not used to limit the scope of the present application.

[0076] 1. Cluster the lane line pixel coordinates in each frame image, and fit lane line segments to each cluster in the clustering result to obtain lane line segment set data containing multiple lane line segment data. The fitting method can be least squares, B-spline, decision tree regression, etc., which is not limited in this application.

[0077] 2. Store multiple lane segment data in the lane segment set data in the first storage list list_lanes; then make a deep copy of the first storage list list_lanes to obtain a second storage list list_lanes_, and the data stored in the initial list_lanes_ is the same as that in list_lanes.

[0078] 3. Traverse the lane segment data stored in each storage position in list_lanes, and perform the following operations on the lane segment data of each storage position in list_lanes, such as the lane segment data list_lanes[i] of the i-th storage position: 1) According to the first reference point A and the last reference point C of list_lanes[i], and the distance head_dis from the preset extension line reference point to the first reference point or the last reference point, obtain the first extension line reference point B and the last extension line reference point D on list_lanes[i], and mark A, B and C, D according to the direction of list_lanes[i]. Specifically: if A to B is east-west (the absolute value of the angle between list_lanes[i] and the north direction is greater than 45°, it is east-west) , then A is marked as P1 and B is marked as P2; if A to B is in the north-south direction (the absolute value of the angle between list_lanes[i] and the north direction is less than 45°, which is the north-south direction), then A is marked as P2 and B is marked as P1; if the direction from C to D is in the north-south direction, then C is marked as P2' and D is marked as P1', and the coordinates of P1 and P2 are (x1_, y1_) and (x2_, y2_) respectively, and the coordinates of P2' and P1' are (xx2_, yy2_) and (xx1_, yy1_) respectively.

[0079] 2) Generate the extension lines extend_shaPely_line and extend_shaPely_line1 at both ends of list_lanes[i] according to [(x1_, y1_), (x2_, y2_)], [(xx2_, yy2_), (xx1_, yy1_)] and the preset horizontal length of the extension line extend_dis[x] and the preset vertical length of the extension line extend_dis[y]. The coordinates of the head or tail of the extension line are as follows: (P2[0]+EXTRAPOL_RATIO[x]*(P2[0]-P1[0]), P2[1]+EXTRAPOL_RATIO[y]*(P2[1]-P1[1])) Where P2[0] is the x coordinate of point P2 or P2', P1[0] is the x coordinate of point P1 or P1', P2[1] is the y coordinate of point P2 or P2', and P1[1] is the y coordinate of point P1 or P1'; EXTRAPOL_RATIO[x] = extend_dis[x] / dis[x], where dis[x] refers to the lateral distance between points P1 and P2 or the lateral distance between points P1' and P2'; EXTRAPOL_RATIO[y] = extend_dis[y] / dis[y], where dis[y] refers to the longitudinal distance between points P1 and P2 or the longitudinal distance between points P1' and P2'.

[0080] 4. For list_lanes[i], traverse the remaining list_lanes backwards starting from the i+1th position.

[0081] The following takes list_lanes[j] as an example to illustrate how to calculate whether list_lanes[i] and list_lanes[j] meet the merging condition, where i and j represent the positions of list_lanes[i] and list_lanes[j] in list_lanes respectively.

[0082] 1) First, obtain the shortest distance min_dis between list_lanes[j] and extend_shaPely_line, and the shortest distance min_dis1 between list_lanes[j] and extend_shaPely_line1. If the minimum value of min_dis and min_dis1 is less than the set threshold ePs, enter the following process. If not, it means that the merging condition is not met and the following process is skipped directly.

[0083] 2) Same as step 3, obtain the extension lines line_t and line_t1 at both ends of list_lanes[j].

[0084] 3) Calculate the shortest distances dis_0 and dis_00 between line_t and line_t1 and extend_shaPely_line respectively, as well as the shortest distances dis_1 and dis_11 between line_t and line_t1 and extend_shaPely_line1 respectively.

[0085] 4) According to the minimum value of dis_0 and dis_00, select the corresponding one of line_t and line_t1 as line_tmP, and according to the minimum value of dis_1 and dis_11, select the corresponding one of line_t and line_t1 as line_tmP1.

[0086] 5) Calculate the angle angle between line_tmP and extend_shapely_line, and the angle angle1 between line_tmP_1 and extend_shapely_line_1. In order to facilitate calculation, the embodiment of the present application converts the angle to 0-90°, and takes angle as an example to illustrate the angle conversion method: If angle>90° and angle<180°, convert angle to 180°~angle; If angle≥90° and angle<270°, convert angle to angle~180°; If angle ≥ 270° and angle ≤ 360°, convert angle to 360°~angle.

[0087] 6) If one of angle or angle1 is smaller than the set threshold, it means that list_lanes[j] can be merged with list_lanes[i].

[0088] Furthermore, the embodiment of the present application takes the case where five short lines line_1, line_2, line_3, line_4, and line_5 are stored in list_lanes, and line_1, line_3, and line_5 need to be merged into one line as an example to illustrate the method for processing lane segments in the embodiment of the present application: Initially, list_lanes and list_lanes_ are the same, see Figure 4 and Figure 5(a); Take line_1 in list_lanes as the current lane segment to be processed, traverse the remaining lane segments, and according to the above steps, determine that line_3 and line_1 meet the preset merging condition, then mark the position of line_3 in list_lanes as 1, and link line_3 to the position of line_1, see Figure 5 (b); Take line_3 in list_lanes as the current lane segment to be processed, traverse the remaining lane segments, and follow the above steps to determine that line_5 and line_3 meet the preset merging conditions. Then mark the position of line_5 in list_lanes as 3, and link line_5 to the position of line_1 based on the index value 1 of the position of line_3, see Figure 5 (c).

[0089] According to list_lanes_, it can be determined that line_1, line_3, and line_5 can be merged into one line to complete a loop. And because line_3 and line_5 are both linked to the position of line_1, the values ​​of the positions of line_3 and line_5 are set values, such as 1. In the next subsequent loop processing, these positions with set values ​​of 1 can be filtered out, and the loop can be repeated from the positions with set values ​​not 1, for example, the next loop processing can be started from line_2 in list_lanes.

[0090] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a lane line processing device, an electronic device and corresponding embodiments.

[0091] Figure 6 It is a schematic diagram of the structure of the lane line processing device shown in an embodiment of the present application.

[0092] See also Figure 6The embodiment of the present application also provides a lane line processing device, including a generation module 610, a calculation module 620, a marking module 630 and an output module 640.

[0093] The generating module 610 is used to generate a first storage list and a second storage list for storing lane segment data sets, wherein the number of storage bits in the first storage list and the second storage list is not less than the total number of lane segments, and one storage bit stores one lane segment data. Preferably, the number of storage bits in the first storage list and the second storage list is the same as the total number of lane segments; and / or, the same storage bit in the first storage list and the second storage list stores one same lane segment.

[0094] The calculation module 620 is used to take the lane segment line_i in the i-th storage position in the first storage list as the lane segment to be processed, and traverse the lane segments in the first storage list from the i+1-th storage position until the lane segment line_j in the j-th storage position that meets the set merging condition with line_i is determined, where j>i, and i=1, 2, 3,..., n-1, and n represents the total number of storage positions in the first storage list that store lane segment data. Specifically, the calculation module 620 is used to: extend at least one end of the current lane line segment line_i to be processed in the first storage list by a first set length to obtain an extension line line_e of at least one end of line_i; traverse the lane line segments in the first storage list from the i+1th storage position, determine the lane line segment whose distance to line_e is less than a set threshold, and use it as the lane line segment to be screened; extend at least one end of the lane line segment to be screened by a second set length to obtain an extension line line_e' of at least one end of the lane line segment to be screened; calculate the angle α between line_e' and line_e, and if α is less than the set angle, use the lane line segment to be screened as the lane line segment line_j.

[0095] The marking module 630 is used to mark the index value of the storage position where the lane segment line_j' corresponding to line_j in the second storage list is located as i', and according to the index value i' of the storage position where line_j' is located, link line_j' to the storage position corresponding to the index value i', wherein i' represents the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located; the calculation module is also used to take the lane segment line_i+1 in the i+1th storage position in the first storage list as the next lane segment to be processed, and repeat the traversal and calculation until the linking of the lane segments in the second storage list and the index value marking of their storage positions are completed. The calculation module is also used to take the lane segment line_i+1 in the i+1th storage position in the first storage list as the next lane segment to be processed, and repeat the traversal and calculation until the linking of the lane segments in the second storage list and the index value marking of their storage positions are completed. Preferably, the marking module 630 is used to directly link line_j' to the storage position corresponding to the index value i' if the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located is not marked with an index value; if the storage position where the lane segment line_i' corresponding to line_i in the second storage list is marked with an index value, link line_j' to the storage position corresponding to the index value of the storage position where line_i' is located.

[0096] The output module 640 is used to output the target lane line data according to the lane line segment data set in the second storage list that has been linked and marked with the index value. Preferably, the output module 640 is used to filter the storage bits marked with the index value in the second storage list, and use the lane line segment data set in the reserved storage bits as the target lane line data.

[0097] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0098] Figure 7 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0099] See also Figure 7 , the electronic device 700 includes a memory 710 and a processor 720 .

[0100] The processor 720 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0101] The memory 710 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. Among them, ROM may store static data or instructions required by the processor 720 or other modules of the computer. The permanent storage may be a readable and writable storage device. The permanent storage may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 710 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 710 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0102] The memory 710 stores executable codes, and when the executable codes are processed by the processor 720 , the processor 720 can execute part or all of the methods described above.

[0103] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0104] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0105] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A lane line processing method, characterized in that: The following steps are involved: S1, generating a first storage list and a second storage list for storing lane segment data sets, wherein one storage bit in the first storage list and one storage bit in the second storage list stores one lane segment data; S2. Take the lane segment line_i in the i-th storage position in the first storage list as the lane segment to be processed, and traverse the lane segments in the first storage list from the i+1-th storage position until the lane segment line_j in the j-th storage position that meets the set merging condition with line_i is determined, where j>i, and i=1, 2, 3, ..., n-1, and n represents the total number of storage positions in the first storage list; S3, marking the index value of the storage position where the lane segment line_j' corresponding to line_j in the second storage list is located as i', and linking line_j' to the storage position corresponding to the index value i', wherein i' represents the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located; S4, taking the lane segment line_i+1 in the i+1th storage position in the first storage list as the next lane segment to be processed, and repeating S2 and S3 until the links of the lane segments in the second storage list and the index value markings of their storage positions are completed; S5. Output target lane line data according to the lane line segment data set in the second storage list that has been linked and marked with index values.

2. The method according to claim 1, characterized in that The number of storage bits in the first storage list and the second storage list is not less than the total number of lane segments.

3. The method according to claim 2, characterized in that The same storage position in the first storage list and the second storage list stores a same lane line segment.

4. The method according to claim 1, characterized in that: Outputting target lane line data according to the lane line segment data set in the second storage list that has completed linking and index value marking comprises the following steps: The storage bits marked with index values ​​in the second storage list are filtered and deleted, and the lane line segment data sets in the retained storage bits are used as target lane line data.

5. The method according to claim 1, characterized in that: The linking of line_j' to the storage bit corresponding to the index value i' comprises the following steps: If the storage position where the lane segment line_i' corresponding to line_i is located in the second storage list is not marked with an index value, directly link line_j' to the storage position corresponding to the index value i'; If the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located is marked with an index value, line_j' is linked to the storage position corresponding to the index value of the storage position where line_i' is located.

6. The method according to claim 1, characterized in that The method of taking the lane segment line_i in the i-th storage position in the first storage list as the lane segment to be processed, and traversing the lane segments in the first storage list from the i+1-th storage position until determining the lane segment line_j in the j-th storage position that satisfies the set merging condition with line_i includes the following steps: Extend at least one end of the current lane line segment line_i to be processed in the first storage list by a first set length to obtain an extension line line_e of at least one end of line_i; Traverse the lane line segments in the first storage list from the i+1th storage position, determine the lane line segment whose distance to line_e is less than the set threshold, and use it as the lane line segment to be screened; Extend at least one end of the lane segment to be screened by a second set length to obtain an extension line line_e' of at least one end of the lane segment to be screened; Calculate the angle α between line_e' and line_e. If α is less than the set angle, take the lane segment to be screened as lane segment line_j.

7. The method according to claim 6, characterized in that: The step of extending at least one end of the current lane line segment line_i to be processed in the first storage list by a first set length to obtain an extension line line_e of at least one end of line_i includes the following steps: According to the slope of the current lane segment line_i to be processed in the first storage list, select the first reference points at the beginning and the end of line_i, and extend the first set length at both ends of line_i based on the first reference points to obtain the extended lines line_e1 and line_e2 at both ends of line_i; and / or, The step of extending at least one end of the lane segment to be screened by a second set length to obtain an extension line line_e' of at least one end of the lane segment to be screened comprises the following steps: According to the slope of the lane segment to be screened, second reference points at the beginning and end of the lane segment to be screened are selected, and based on the second reference points, both ends of the lane segment to be screened are extended by a second set length to obtain extended lines line_e1' and line_e2' at both ends of the lane segment to be screened.

8. The method according to claim 7, characterized in that The step of traversing the lane segments in the first storage list from the i+1th storage position to determine the lane segments whose distance to line_e is less than a set threshold and using them as lane segments to be screened includes the following steps: Traverse the lane segments in the first storage list from the i+1th storage position, and calculate the shortest distance D1 between each lane segment and line_e1, and the shortest distance D2 between each lane segment and line_e2 in turn; Compare the smaller value of D1 and D2 with the set threshold; If the smaller value of D1 and D2 is less than the set threshold, the lane segment is used as the lane segment to be screened.

9. The method according to claim 7, characterized in that: Before calculating the angle α between line_e' and line_e, the following steps are included: Calculate the shortest distances D1' and D1'' between line_e1' and line_e2' and line_e1, and the shortest distances D2' and D2'' between line_e1' and line_e2' and line_e2; Select the extension line corresponding to the smaller value of D1' and D1'' and use it as the first extension line to be screened; The extension line corresponding to the smaller value of D2' and D2'' is selected as the second extension line to be screened.

10. The method according to claim 9, characterized in that The calculating of the angle α between line_e' and line_e, and if α is less than a set angle, taking the lane segment to be screened as lane segment line_j, comprises the following steps: Calculate the angle α1 between the first extended line to be screened and line_e1, and the angle α2 between the second extended line to be screened and line_e2; If α1 and / or α2 is smaller than the set angle, the lane line segment to be screened is used as lane line segment line_j.

11. A lane line processing device, characterized in that: include: A generating module, used to generate a first storage list and a second storage list for storing lane segment data sets, wherein the number of storage bits in the first storage list and the second storage list is not less than the total number of lane segments, and one storage bit stores one lane segment data; a calculation module, configured to use the lane segment line_i in the i-th storage position in the first storage list as the lane segment to be processed, and traverse the lane segments in the first storage list from the i+1-th storage position until determining the lane segment line_j in the j-th storage position that satisfies the set merging condition with line_i, wherein j>i, and i=1, 2, 3, ..., n-1, and n represents the total number of storage positions in the first storage list storing lane segment data; a marking module, used to mark the index value of the storage position where the lane segment line_j' corresponding to line_j in the second storage list is located as i', and link line_j' to the storage position corresponding to the index value i', wherein i' represents the storage position where the lane segment line_i' corresponding to line_i in the second storage list is located; the calculation module is also used to take the lane segment line_i+1 in the i+1th storage position in the first storage list as the next lane segment to be processed, and repeat traversal and calculation until the linking of the lane segments in the second storage list and the index value marking of their storage positions are completed; An output module is used to output target lane line data according to the lane line segment data set in the second storage list that has completed linking and index value marking.

12. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method as claimed in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: An executable code is stored thereon, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as claimed in any one of claims 1 to 10.

Images