Methods, apparatus, equipment, and storage media for detecting lane lines in high-precision maps

Abstract

The present disclosure provides a method, device, equipment and storage medium for detecting lane lines, and relates to the field of artificial intelligence technology, specifically the field of intelligent transportation and autonomous driving technology. The method includes: acquiring a plurality of continuous image frames collected by a vehicle acquisition device; determining a lane line instance corresponding to each of the plurality of continuous image frames; determining the attribute information of the lane line instance based on the instance sub-image of the lane line instance corresponding to each image frame; concatenating the attribute information of the lane line instance corresponding to each image frame to obtain complete lane line information; determining the position of the lane line change point based on the complete lane line information and the information in a pre-constructed database. The method for detecting lane lines provided by the present disclosure can construct the complete information of the current lane line only through the multiple image data collected by the acquisition device, is simple and easy to operate, has low cost, and improves the efficiency of detecting lane line change information.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the field of intelligent transportation and autonomous driving technology, and in particular to methods, devices, equipment and storage media for detecting lane lines on high-precision maps. Background Art

[0002] High-precision maps, also known as high-precision maps, are used by self-driving cars. High-precision maps have accurate vehicle location information and rich road element data information, which can help cars predict complex road information, such as slope, curvature, heading, etc., and better avoid potential risks. Achieving rapid updates of high-precision map elements, solving the timeliness of ground elements, and maintaining the freshness of high-precision map elements are very important parts of high-precision map making and updating. In order to achieve rapid updates of high-precision map elements, "crowdsourcing" vehicles are generally used to collect traffic road images, and real-time feedback is used to extract elements, find the change points of map elements, and then update the map elements. Among all map elements, lane lines are an important part of the rapid update of map elements. In the prior art, visual images with internal and external parameters and positioning data are generally used to detect lane line information, which has high equipment costs; in addition, full-volume differential based on vision is prone to false detection in the case of occlusion and light changes. Summary of the invention

[0003] The present disclosure provides a method, apparatus, device and storage medium for detecting lane lines.

[0004] According to a first aspect of the present disclosure, a method for detecting lane lines is provided, comprising: acquiring a plurality of consecutive image frames acquired by an acquisition device of a vehicle; determining a lane line instance corresponding to each of the plurality of consecutive image frames; determining attribute information of the lane line instance based on an instance sub-image of the lane line instance corresponding to each image frame; concatenating the attribute information of the lane line instance corresponding to each image frame to obtain complete lane line information; and determining a position of a change point of the lane line based on the complete lane line information and information in a pre-constructed database.

[0005] According to a second aspect of the present disclosure, there is provided a device for detecting lane lines, comprising: an acquisition module, configured to acquire a plurality of consecutive image frames acquired by an acquisition device of a vehicle; a first determination module, configured to determine a lane line instance corresponding to each of the plurality of consecutive image frames; a second determination module, configured to determine attribute information of the lane line instance based on an instance sub-image of the lane line instance corresponding to each image frame; a concatenation module, configured to concatenate the attribute information of the lane line instance corresponding to each image frame to obtain complete lane line information; and a third determination module, configured to determine the position of a change point of the lane line based on the complete lane line information and information in a pre-constructed database.

[0006] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation manner in the first aspect.

[0007] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method described in any implementation manner of the first aspect.

[0008] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the computer program implements the method described in any implementation manner in the first aspect.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is an exemplary system architecture diagram in which the present disclosure may be applied;

[0012] Figure 2 is a flow chart of an embodiment of a method for detecting lane lines according to the present disclosure;

[0013] Figure 3 is a flowchart of another embodiment of a method for detecting lane lines according to the present disclosure;

[0014] Figure 4 is a flow chart of another embodiment of a method for detecting lane lines according to the present disclosure;

[0015] Figure 5 is a schematic diagram of an application scenario of the method for detecting lane lines according to the present disclosure;

[0016] Figure 6 is a schematic structural diagram of an embodiment of a device for detecting lane lines according to the present disclosure;

[0017] Figure 7 It is a block diagram of an electronic device used to implement the method for detecting lane lines according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0019] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the method for detecting lane lines or the apparatus for detecting lane lines of the present disclosure can be applied.

[0021] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0022] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send information, etc. Various client applications can be installed on the terminal devices 101, 102, 103.

[0023] The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the terminal devices 101, 102, 103 are software, they may be installed in the above electronic devices. They may be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0024] The server 105 can provide various services. For example, the server 105 can analyze and process multiple image frames obtained from the terminal devices 101, 102, and 103, and generate processing results (such as the position of the change point of the lane line).

[0025] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0026] It should be noted that the method for detecting lane lines provided in the embodiments of the present disclosure is generally executed by the server 105 , and accordingly, the device for detecting lane lines is generally disposed in the server 105 .

[0027] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0028] Continue to refer Figure 2 , which shows a process 200 of an embodiment of a method for detecting lane lines according to the present disclosure. The method for detecting lane lines comprises the following steps:

[0029] Step 201: Acquire a plurality of continuous image frames captured by a capture device of the vehicle.

[0030] In this embodiment, the execution subject of the method for detecting lane lines (for example Figure 1 The server 105 shown in the figure can obtain multiple continuous image frames collected by the vehicle's collection device. Among them, the vehicle can be a vehicle participating in crowdsourcing, that is, the image data of the road (also referred to as road video data) can be collected in a crowdsourcing mode. Specifically, a large number of low-cost ordinary vehicles in society (these vehicles are called crowdsourcing vehicles) are involved in road video collection. During normal driving on the road, the crowdsourcing vehicles use the on-board collection device to collect video of the current road, and upload the collected video data so that the above-mentioned execution subject can obtain the collected multiple image frames. It can be understood that from a macro perspective, there is a steady stream of crowdsourcing vehicles traveling on various roads in the road network. By adopting the crowdsourcing mode to collect road videos, when any road in the road network changes, the road change information can be discovered in a timely manner. On the one hand, the timeliness and comprehensiveness of the road change information are guaranteed, and on the other hand, the collection cost can be reduced compared with professional collection vehicles. In addition, the vehicle can also be a professional collection vehicle, which is generally equipped with collection devices such as laser radar and cameras. It can shoot the current road through the collection device and upload the collected continuous image frames so that the above-mentioned execution entity can obtain the collected multiple image frames.

[0031] Among them, the acquisition device on the vehicle can capture the road while the vehicle is driving to obtain image frames. The acquisition device can be installed at any position of the vehicle as long as it can capture the road.

[0032] It should be noted that the vehicle's collection device generally also has a GPS (Global Positioning System) device, that is, the vehicle's collection device will also collect the vehicle's geographic location information in real time. The above-mentioned execution entity will also obtain the collected geographic location information and match the collected image frames with the collected geographic location information, so that each image frame has corresponding geographic location information.

[0033] Step 202: Determine a lane line instance corresponding to each image frame in a plurality of consecutive image frames.

[0034] In this embodiment, the execution subject can determine the lane line instance corresponding to each of the multiple image frames collected in step 201. Specifically, the execution subject will process each of the multiple consecutive image frames collected respectively, so as to obtain the lane line instance corresponding to each image frame. For example, the execution subject can perform instance segmentation on each image frame respectively, and then perform image in-image clustering on the segmentation results of each image frame, so as to obtain the lane line instance corresponding to the image frame, and the lane line instance is indexed from the leftmost lane line, and is marked from left to right as lane line 1, lane line 2, lane line 3... and so on.

[0035] Step 203: Determine attribute information of the lane line instance based on the instance sub-image of the lane line instance corresponding to each image frame.

[0036] In this embodiment, the execution subject may determine the instance sub-image of the lane line instance corresponding to each image frame obtained in step 202, and then determine the attribute information of the lane line instance based on the instance sub-image. For example, the execution subject may segment the lane line instance corresponding to each image frame, and obtain multiple instance sub-images of the lane line instance through segmentation; then classify each instance sub-image to obtain the attribute information of the lane line instance corresponding to each instance sub-image; finally, based on the attribute information of the lane line instance corresponding to each instance sub-image, the attribute information of the lane line instance corresponding to the image frame may be obtained. The attribute information includes, but is not limited to, the color information of the lane line, the virtuality and reality information of the lane line, and the thickness information of the lane line.

[0037] Step 204 , concatenate the attribute information of the lane line instance corresponding to each image frame to obtain complete lane line information.

[0038] In this embodiment, the above-mentioned execution subject can concatenate the attribute information of the lane line instance corresponding to each image frame to obtain complete lane line information. In this embodiment, the lane line attribute information corresponding to a single image frame reflects the attribute information of the current lane line at a certain geographical location in the road to be detected (i.e., the geographical location captured by the image frame). Since the lane line information of any geographical location of the road to be detected is recorded in multiple consecutive image frames, the attribute information of the lane line instance corresponding to each image frame is concatenated to obtain the complete information of the current lane line of the road to be detected.

[0039] It should be noted that, in some embodiments, after concatenating the attribute information of the lane line instances corresponding to the image frames, the above-mentioned execution entity will also cluster the concatenation results, thereby interpolating and completing the information of the lane lines that are blocked or not detected due to factors such as light, so as to make the obtained lane line information more complete and accurate.

[0040] Step 205 , based on the complete lane line information and the information in the pre-built database, determine the location of the lane line change point.

[0041] In this embodiment, the execution subject may determine the position of the lane line change point based on the complete lane line information obtained in step 204 and the information in the pre-built database. For example, the execution subject may first determine the attribute change position of the lane line on the image side based on the complete lane line information, thereby obtaining a change node graph corresponding to the attribute change position. Then, the execution subject may determine the attribute change position of the lane line in the database based on the pre-built database containing historical lane line information, thereby obtaining a change node graph corresponding to the attribute change position. It can be understood that for each lane line in the road, the attributes of the lane line may change at certain geographical locations of the road, for example: from a dotted line to a solid line (or from a solid line to a dotted line), from a yellow line to a white line (or from a white line to a yellow line), from a thick line to a thin line (or from a thin line to a thick line), etc. Therefore, for a lane line in an image frame, a certain attribute of the lane line may change or may not change. Finally, the execution subject may perform a difference between the two change node graphs to determine the position of the lane line change point. Specifically, the change point may include a number change point, a color change point, a virtual-real change point, a thickness change point, etc. The above-mentioned execution entity can determine information such as changes in the number, color, virtuality, and thickness of lane lines.

[0042] The method for detecting lane lines provided by the disclosed embodiment first obtains a plurality of continuous image frames collected by the vehicle's acquisition device; and determines the lane line instance corresponding to each of the plurality of continuous image frames; then determines the attribute information of the lane line instance based on the instance sub-image of the lane line instance corresponding to each image frame; then concatenates the attribute information of the lane line instance corresponding to each image frame to obtain complete lane line information; and finally determines the position of the lane line change point based on the complete lane line information and the information in the pre-constructed database. The method for detecting lane lines in this embodiment can construct the complete information of the current lane line only through the image data collected by the acquisition device without other parameters, and determines the position of the lane line change point based on the complete lane line information and the information in the pre-constructed database. The method is simple and easy to operate, improves the efficiency of detecting lane line change point information, and reduces the cost of detecting lane line change point information. At the same time, it can also solve the false detection when the lane line is unclear due to occlusion and light change; in addition, the lane line change information detected by this embodiment is used to update the data of the high-precision map, which can ensure the freshness and accuracy of the high-precision map.

[0043] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0044] Continue to refer Figure 3 , Figure 3 A process 300 of another embodiment of a method for detecting lane lines according to the present disclosure is shown. The method for detecting lane lines comprises the following steps:

[0045] Step 301: Acquire a plurality of continuous image frames captured by a capture device of the vehicle.

[0046] Step 301 is basically the same as step 201 of the aforementioned embodiment. The specific implementation method can refer to the aforementioned description of step 201, which will not be repeated here.

[0047] Step 302 : For each of the multiple consecutive image frames, perform instance segmentation on the image frame to obtain a segmentation result.

[0048] In this embodiment, for each of the multiple consecutive image frames, the execution subject of the method for detecting lane lines (for example Figure 1 The server 105 shown in the figure can perform instance segmentation on the image frame and obtain corresponding segmentation results. It should be noted that the instance segmentation method can be implemented using existing technologies, which will not be described in detail here.

[0049] Step 303: cluster the segmentation results to obtain lane line instances corresponding to the image frames.

[0050] In this embodiment, the execution subject can cluster the segmentation results obtained in step 302 to obtain the lane line instance corresponding to the image frame. The lane line instance is indexed from the leftmost lane line, and is marked from left to right as lane line 1, lane line 2, lane line 3, and so on. Based on the above steps, the lane line instance corresponding to each image frame in multiple consecutive image frames can be obtained, thereby improving the accuracy of the obtained lane line information.

[0051] Step 304 , segment the lane line instance corresponding to each image frame to obtain at least one corresponding instance sub-image.

[0052] In this embodiment, the execution entity may segment the lane line instance corresponding to each image frame respectively, so as to obtain at least one instance sub-image corresponding to each image frame.

[0053] In some optional implementations of this embodiment, step 304 includes: determining the sampling position of each image frame based on the position of two adjacent groups of lane lines in the lane line instance; calculating the segmentation point of the image frame at the sampling position to obtain at least one corresponding instance sub-image. Since the above-mentioned execution subject also obtains the collected geographic location information and matches the collected image frame with the collected geographic location information, each image frame has corresponding geographic location information. Therefore, the above-mentioned execution subject will calculate the segmentation point at different sampling positions based on the actual position between the two adjacent groups of lane lines in the lane line instance. In this implementation, the attribute of the same frame image is defined as one, and its accuracy depends on the distance between frames and is related to the frame rate of the acquisition. Therefore, the above-mentioned execution subject will calculate the segmentation point at different sampling positions to obtain at least one corresponding instance sub-image. Thereby solving the segmentation of curved lane lines.

[0054] Step 305: Use a classifier to classify each instance sub-image in at least one instance sub-image.

[0055] In this embodiment, the above-mentioned execution entity can use the classifier to classify each instance sub-image in at least one instance sub-image respectively, that is, each instance sub-image in at least one instance sub-image is input into the classifier respectively, and the classification result of each instance sub-image can be output.

[0056] Step 306: Determine the attribute information of the lane line instance based on the classification result.

[0057] In this embodiment, the execution subject may determine the attribute information of the lane line instance based on the classification result of step 305, wherein the attribute information includes at least one of the following: virtuality attribute, color attribute, thickness attribute. For example, the attribute information of the lane line instance may be: dotted line, yellow line, thick line.

[0058] By segmenting the lane line instance to obtain the corresponding instance sub-image, and classifying the instance sub-image, the attribute information of the lane line instance is obtained, which can improve the accuracy of the attribute information of the obtained lane line instance.

[0059] Step 307 , concatenate the attribute information of the lane line instance corresponding to each image frame to obtain complete lane line information.

[0060] Step 308 , based on the complete lane line information and the information in the pre-built database, determine the location of the lane line change point.

[0061] Steps 307-308 are basically consistent with steps 204-205 of the aforementioned embodiment. For specific implementation methods, reference may be made to the aforementioned description of steps 204-205, which will not be repeated here.

[0062] from Figure 3 It can be seen that Figure 2 Compared with the corresponding embodiments, the method for detecting lane lines in this embodiment highlights the steps of determining the lane line instance corresponding to each image frame and determining the attribute information of the lane line instance, thereby improving the accuracy of the attribute information of the obtained lane line instance.

[0063] Continue to refer Figure 4 , Figure 4 A process 400 of another embodiment of a method for detecting lane lines according to the present disclosure is shown. The method for detecting lane lines comprises the following steps:

[0064] Step 401, obtaining a plurality of continuous image frames captured by a capture device of the vehicle.

[0065] Step 402 : For each of the multiple consecutive image frames, perform instance segmentation on the image frame to obtain a segmentation result.

[0066] Step 403: cluster the segmentation results to obtain lane line instances corresponding to the image frames.

[0067] Step 404 , segment the lane line instance corresponding to each image frame to obtain at least one corresponding instance sub-image.

[0068] Step 405: Classify each instance sub-image in at least one instance sub-image using a classifier.

[0069] Step 406: Determine attribute information of the lane instance based on the classification result.

[0070] Steps 401-406 are basically consistent with steps 301-306 of the aforementioned embodiment. For specific implementation methods, reference may be made to the aforementioned description of steps 301-306, which will not be repeated here.

[0071] Step 407 , based on the geographical location information corresponding to each image frame, the attribute information of the lane line instances corresponding to a plurality of consecutive image frames are concatenated.

[0072] In this embodiment, the execution subject of the method for detecting lane lines (for example Figure 1 The server 105 shown in the figure can concatenate the attribute information of the lane line instances corresponding to the multiple image frames based on the geographic location information of each image frame, so as to obtain the concatenated lane line. Since each image frame has corresponding geographic location information, the above-mentioned execution subject can concatenate the attribute information of the lane line instances corresponding to the multiple continuous image frames according to the geographic location information to form a very long lane line.

[0073] Step 408: cluster the concatenated results to obtain complete lane line information.

[0074] In this embodiment, the execution subject can cluster the concatenation results obtained in step 407 to obtain complete lane line information. By clustering the concatenation results of the attribute information of the lane line instances of multiple image frames, the lane lines that are not detected due to factors such as occlusion and light changes can be interpolated and completed, thereby completing the lane lines occluded by vehicles and the lane lines that are unclear due to light changes, so that the obtained lane information is more complete and accurate.

[0075] Step 409 , determining a first change node graph corresponding to a lane line attribute change position based on the complete lane line information.

[0076] In this embodiment, the execution subject can determine the attribute change position of the lane line based on the complete lane line obtained in step 408, thereby obtaining a change node graph corresponding to the attribute change position, which is recorded as a first change node graph. It can be understood that for each lane line in the road, the attributes of the lane line may change at certain geographical locations of the road, for example: from a dotted line to a solid line (or from a solid line to a dotted line), from a yellow line to a white line (or from a white line to a yellow line), from a thick line to a thin line (or from a thin line to a thick line), etc. Therefore, for a lane line in an image frame, a certain attribute of the lane line may change or may not change.

[0077] In this embodiment, when the execution subject detects that the attribute of the lane line has changed, it will determine the first change node graph corresponding to the attribute change position.

[0078] Step 410: Based on the information of the lane line in the pre-constructed database, determine a second change node graph corresponding to the attribute change position of the lane line in the database.

[0079] In this embodiment, since a database containing historical lane line information has been pre-built, the above-mentioned execution entity will obtain the historical information of the lane line in the database, and determine the attribute change position of the lane line based on the historical information. The attribute change here may also be from a dotted line to a solid line (or from a solid line to a dotted line), from a yellow line to a white line (or from a white line to a yellow line), from a thick line to a thin line (or from a thin line to a thick line), etc., and then the above-mentioned execution entity will determine the change node graph corresponding to the attribute change position and record it as the second change node graph.

[0080] Step 411, performing a difference between the first change node graph and the second change node graph to obtain the change point position of the lane line.

[0081] In this embodiment, the above-mentioned execution subject can differentiate the first change node graph determined in step 409 from the second change node graph determined in step 410 to obtain the change point position of the lane line. Differentiation is also known as differential function or differential operation. The result of the difference reflects a change between discrete quantities and is a tool for studying discrete mathematics. The above-mentioned execution subject can obtain the change point position of the current lane line and the historical lane line stored in the database by differentiating the first change node graph and the second change node graph. Specifically, the change point may include number change point, color change point, virtual-real change point, thickness change point, etc.

[0082] Through the above steps, the attribute change position of the lane line on the image side and the attribute change position of the lane line on the database side can be obtained, so as to determine the final lane line change point position, thereby improving the accuracy of the obtained lane line change point position.

[0083] from Figure 4 It can be seen that Figure 3 Compared with the corresponding embodiments, the method for detecting lane lines in this embodiment highlights the step of clustering the concatenation results of multiple image frames to obtain complete lane line information, and the step of aligning the lane line information with the information in a pre-built database to obtain the position of the lane line change point. In this way, lane lines that are not detected due to factors such as occlusion and light changes can be interpolated and completed, making the obtained lane information more complete and accurate, thereby improving the accuracy of the obtained lane line change point position.

[0084] Continue to see Figure 5 , which shows a schematic diagram of an application scenario of the method for detecting lane lines according to the present disclosure.

[0085] exist Figure 5 In the application scenario, first, the above-mentioned execution subject will obtain an image sequence captured by the vehicle's acquisition device, that is, multiple continuous image frames.

[0086] Then, lane line AI (Artificial Intelligence) recognition is performed on the collected image sequence, specifically including: 1) lane line detection, that is, for each image frame in the image sequence, the image frame is segmented to obtain a segmentation result; the segmentation results are concatenated to obtain a lane line instance corresponding to the image frame; 2) lane line attribute recognition, that is, the lane line instance corresponding to each image frame is segmented to obtain at least one corresponding instance sub-image, and each instance sub-image in the at least one instance sub-image is classified by a classifier, and the attribute information of the lane line instance is determined based on the classification result.

[0087] After that, the visual change point node graph on the image side is determined, specifically including: concatenating the results corresponding to multiple image frames between frames, and then clustering the concatenated results to obtain complete lane line information. Then, based on the complete lane line information, the first change node graph corresponding to the attribute change position of the lane line on the image side is determined. And the second change node graph corresponding to the attribute information change position of the lane line on the database side is determined.

[0088] Finally, the first change node graph is differentiated from the second change node graph to obtain the change point position of the lane line, wherein the change point can be a number change point, a virtuality change point, a color change point, a thickness change point, etc.

[0089] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for detecting lane lines. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0090] like Figure 6As shown, the lane line detection device 600 of this embodiment includes: an acquisition module 601, a first determination module 602, a second determination module 603, a concatenation module 604 and a third determination module 605. Among them, the acquisition module 601 is configured to acquire a plurality of continuous image frames acquired by the acquisition device of the vehicle; the first determination module 602 is configured to determine the lane line instance corresponding to each of the plurality of continuous image frames; the second determination module 603 is configured to determine the attribute information of the lane line instance based on the instance sub-image of the lane line instance corresponding to each image frame; the concatenation module 604 is configured to concatenate the attribute information of the lane line instance corresponding to each image frame to obtain complete lane line information; the third determination module 605 is configured to determine the position of the change point of the lane line based on the complete lane line information and the information in the pre-constructed database.

[0091] In the present embodiment, in the lane line detection device 600, the specific processing of the acquisition module 601, the first determination module 602, the second determination module 603, the serial connection module 604 and the third determination module 605 and the technical effects thereof can be referred to respectively. Figure 2 The relevant descriptions of steps 201 - 205 in the corresponding embodiment are not repeated here.

[0092] In some optional implementations of the present embodiment, the first determination module includes: a segmentation submodule, configured to perform instance segmentation on each image frame in a plurality of consecutive image frames to obtain a segmentation result; and a first clustering submodule, configured to cluster the segmentation results to obtain a lane line instance corresponding to the image frame.

[0093] In some optional implementations of the present embodiment, the second determination module includes: a segmentation submodule, configured to segment the lane line instances corresponding to each image frame respectively to obtain at least one corresponding instance sub-image; a classification submodule, configured to use a classifier to classify each instance sub-image in at least one instance sub-image respectively; and a first determination submodule, configured to determine the attribute information of the lane line instance based on the classification result, wherein the attribute information includes at least one of the following: virtuality and reality attributes, color attributes, and thickness attributes.

[0094] In some optional implementations of this embodiment, the segmentation submodule includes: a determination unit, configured to determine the sampling position of each image frame based on the positions of two adjacent groups of lane lines in the lane line instance; and a calculation unit, configured to calculate the segmentation point of the image frame at the sampling position to obtain at least one corresponding instance sub-image.

[0095] In some optional implementations of the present embodiment, the concatenation module includes: a concatenation sub-module, configured to concatenate the attribute information of the lane line instances corresponding to multiple consecutive image frames based on the geographic location information corresponding to each image frame; and a second clustering sub-module, configured to cluster the concatenation results to obtain complete lane line information.

[0096] In some optional implementations of this embodiment, the third determination module includes: a second determination submodule, configured to determine a first change node graph corresponding to the attribute change position of the lane line based on the complete lane line information; a third determination submodule, configured to determine a second change node graph corresponding to the attribute change position of the lane line in the database based on information in a pre-built database; a differentiation submodule, configured to differentiate the first change node graph from the second change node graph to obtain the change point position of the lane line.

[0097] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0098] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0099] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0100] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0101] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a method for detecting lane lines. For example, in some embodiments, the method for detecting lane lines may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the method for detecting lane lines described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform a method for detecting lane lines in any other appropriate manner (e.g., by means of firmware).

[0102] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0104] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0107] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0108] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0109] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for detecting lane lines, comprising: Acquire a plurality of continuous image frames captured by a capture device of the vehicle; Determining a lane line instance corresponding to each image frame in the plurality of consecutive image frames; Determining attribute information of the lane line instance based on the instance sub-image of the lane line instance corresponding to each image frame; Concatenating the attribute information of the lane line instances corresponding to each image frame to obtain complete lane line information; Determine the position of the lane line change point based on the complete lane line information and information in a pre-built database; Wherein, determining the attribute information of the lane line instance based on the instance sub-image of the lane line instance corresponding to each image frame includes: Determine a sampling position of each image frame based on positions of two adjacent groups of lane lines in the lane line instance; Calculate the segmentation points of the image frame at the sampling position to obtain at least one corresponding instance sub-image; Using a classifier to classify each instance sub-image in the at least one instance sub-image respectively; Attribute information of the lane line instance is determined based on the classification result, wherein the attribute information includes at least one of the following: virtuality attribute, color attribute, and thickness attribute.

2. The method according to claim 1, wherein: The determining of a lane line instance corresponding to each image frame in the plurality of consecutive image frames includes: For each image frame in the plurality of consecutive image frames, performing instance segmentation on the image frame to obtain a segmentation result; The segmentation results are clustered to obtain lane line instances corresponding to the image frames.

3. The method according to any one of claims 1 to 2, wherein: The attribute information of the lane line instance corresponding to each image frame is concatenated to obtain complete lane line information, including: Based on the geographic location information corresponding to each image frame, concatenating the attribute information of the lane line instances corresponding to the plurality of consecutive image frames; Cluster the concatenated results to obtain complete lane line information.

4. The method according to any one of claims 1 to 3, wherein: The determining the position of the lane line change point based on the complete lane line information and information in a pre-built database includes: Determine a first change node graph corresponding to a lane line attribute change position based on the complete lane line information; Based on the information in the pre-built database, determine a second change node graph corresponding to the attribute change position of the lane line in the database; The first change node graph is differentiated from the second change node graph to obtain the change point position of the lane line.

5. A device for detecting lane lines, comprising: An acquisition module is configured to acquire a plurality of continuous image frames acquired by an acquisition device of the vehicle; A first determination module is configured to determine a lane line instance corresponding to each image frame in the plurality of consecutive image frames; A second determination module is configured to determine attribute information of the lane line instance based on the instance sub-image of the lane line instance corresponding to each image frame; A concatenation module is configured to concatenate the attribute information of the lane line instances corresponding to each image frame to obtain complete lane line information; A third determination module is configured to determine the position of the change point of the lane line based on the complete lane line information and information in a pre-built database; Wherein, the second determining module is further configured to: Determine a sampling position of each image frame based on positions of two adjacent groups of lane lines in the lane line instance; Calculate the segmentation points of the image frame at the sampling position to obtain at least one corresponding instance sub-image; Using a classifier to classify each instance sub-image in the at least one instance sub-image respectively; Attribute information of the lane line instance is determined based on the classification result, wherein the attribute information includes at least one of the following: virtuality attribute, color attribute, and thickness attribute.

6. The device according to claim 5, wherein: The first determining module comprises: A segmentation submodule is configured to perform instance segmentation on each image frame in the plurality of consecutive image frames to obtain a segmentation result; The first clustering submodule is configured to cluster the segmentation results to obtain lane line instances corresponding to the image frames.

7. The device according to any one of claims 5 to 6, wherein: The serial connection module comprises: A concatenation submodule is configured to concatenate the attribute information of the lane line instances corresponding to the plurality of consecutive image frames based on the geographical location information corresponding to each image frame; The second clustering submodule is configured to cluster the concatenation results to obtain complete lane line information.

8. The device according to any one of claims 5 to 7, wherein: The third determination module comprises: A second determination submodule is configured to determine a first change node graph corresponding to a lane line attribute change position based on the complete lane line information; A third determination submodule is configured to determine, based on information in a pre-built database, a second change node graph corresponding to the attribute change position of the lane line in the database; The difference submodule is configured to differentiate the first change node graph from the second change node graph to obtain the change point position of the lane line.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 4.

11. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 4.

Images