Lane line detection method and device, storage medium, processor and electronic device

Abstract

The application discloses a lane line detection method and device, a storage medium, a processor and an electronic device. The method comprises the following steps: acquiring an image to be recognized, wherein the display content of the image to be recognized comprises lane lines to be recognized; analyzing the image to be recognized by using a first neural network model to obtain a first segmentation graph, wherein the first neural network model is used for full-image segmentation of the image to be recognized, and the first segmentation graph is a full-image non-connected segmentation graph; analyzing the image to be recognized by using a second neural network model to obtain a second segmentation graph, wherein the second neural network model is used for lane line segmentation, and the second segmentation graph is a full-image connected segmentation graph; and determining a classification result of the lane lines based on the first segmentation graph and the second segmentation graph. The application solves the technical problem that the vehicle motion control capability is limited due to the fact that the advanced auxiliary driving system cannot distinguish the virtual and real information of the lane lines during the driving process of the vehicle.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically, to a lane line detection method, apparatus, storage medium, processor, and electronic device. Background Technology

[0002] With the rapid development of related science and technology and the continuous increase in car ownership, car safety has become one of the key concerns in modern life. As a result, more and more cars are equipped with various advanced driver assistance systems, among which lane detection is one of the most important technologies.

[0003] Vision-based lane detection methods are widely used due to their rich information provision and low cost. Vision-based lane detection methods can generally be divided into two categories:

[0004] (1) Feature method: Use the feature information such as lane line color, width or gradient change to extract lane lines from the road surface area, or use the texture information of the road surface combined with the region growth and classification methods to obtain the entire road area.

[0005] (2) Model method: The lane is abstracted into an appropriate geometric model. First, the edge detection method is used to obtain the edge information of the lane line or road. Then, the parameters of the geometric model are obtained by methods such as Random Sample Consensus (RANSAC), least squares method, and Hough transform to fit the corresponding lane line.

[0006] These existing lane detection methods can perform preliminary lane detection, but they cannot distinguish between solid and dashed lane lines. However, accurate identification of solid and dashed lane line information is crucial during advanced driver assistance systems (ADAS) control of vehicles, especially when the vehicle intends to change lanes. In other words, a key issue during vehicle operation is that existing ADAS lane detection technologies cannot differentiate between solid and dashed lane line information, thus limiting the improvement of vehicle motion control capabilities.

[0007] There is currently no effective solution to the above problems. Summary of the Invention

[0008] This invention provides a lane line detection method, apparatus, storage medium, processor, and electronic device to at least solve the technical problem that the improvement of vehicle motion control capability is limited due to the inability of advanced driver assistance systems to distinguish between real and virtual lane line information during vehicle operation.

[0009] According to one embodiment of the present invention, a lane line detection method is provided, comprising:

[0010] The process involves: acquiring an image to be identified, wherein the displayed content of the image includes lane lines to be identified; analyzing the image to be identified using a first neural network model to obtain a first segmentation map, wherein the first neural network model is used to segment the entire image to be identified, and the first segmentation map is a segmentation map that is not connected to the entire image; analyzing the image to be identified using a second neural network model to obtain a second segmentation map, wherein the second neural network model is used to segment the lane lines, and the second segmentation map is a segmentation map that is connected to the entire image; and determining the classification result of the lane lines based on the first segmentation map and the second segmentation map.

[0011] Optionally, the first neural network model is trained using multiple sets of image data through machine learning. Each set of image data includes: the original image and the categories of the full-image semantic segmentation labeled in the original image.

[0012] Optionally, the second neural network model is trained using multiple sets of image data through machine learning. Each set of image data includes: the original image and the lane lines marked in the original image.

[0013] Optionally, determining the lane line classification result based on the first segmentation map and the second segmentation map includes: projecting the first segmentation map and the second segmentation map onto the same target coordinate system; calculating the ratio of lane line lengths at corresponding positions in the first segmentation map and the second segmentation map in the target coordinate system to obtain the calculation result; and determining the lane line classification result based on the comparison result of the calculation result and a preset threshold.

[0014] Optionally, calculating the ratio of lane line lengths at corresponding positions in the first and second segmented images to obtain the calculation result includes: obtaining the first starting point coordinate information, first marker information, and first sequence number information of the object to be calculated from the first segmented image, and obtaining the second starting point coordinate information, second marker information, and second sequence number information of the object to be calculated from the second segmented image; when it is determined through the first and second marker information that the category of the object to be calculated is lane line, and it is determined through the first and second sequence number information that the object to be calculated is located at the corresponding position in the first and second segmented images, the ratio of lane line lengths of the object to be calculated is obtained using the first and second starting point coordinate information to obtain the calculation result.

[0015] Optionally, based on the comparison between the calculation result and the preset threshold, the classification result of the lane line is determined as follows: when the calculation result is greater than the preset threshold, the lane line is classified as a solid line; when the calculation result is less than or equal to the preset threshold, the lane line is classified as a dashed line.

[0016] According to one embodiment of the present invention, a lane line detection device is also provided, comprising:

[0017] The system comprises: an acquisition module for acquiring an image to be recognized, wherein the displayed content of the image includes lane lines to be recognized; a first analysis module for analyzing the image to be recognized using a first neural network model to obtain a first segmentation map, wherein the first neural network model is used to perform full-image segmentation of the image to be recognized, and the first segmentation map is a segmentation map that is not connected to the entire image; a second analysis module for analyzing the image to be recognized using a second neural network model to obtain a second segmentation map, wherein the second neural network model is used to segment the lane lines, and the second segmentation map is a segmentation map that is connected to the entire image; and a detection module for determining the classification result of the lane lines based on the first and second segmentation maps.

[0018] Optionally, the first neural network model in the first analysis module is trained using multiple sets of image data through machine learning. Each set of image data includes: the original image and the categories of the full-image semantic segmentation labeled in the original image.

[0019] Optionally, the second neural network model in the second analysis module is trained using multiple sets of image data through machine learning. Each set of image data includes: the original image and the lane lines marked in the original image.

[0020] Optionally, the detection module includes: a projection unit for projecting the first segmentation image and the second segmentation image onto the same target coordinate system; a calculation unit for calculating the ratio of lane line lengths at corresponding positions in the first segmentation image and the second segmentation image in the target coordinate system to obtain a calculation result; and a determination unit for determining the classification result of the lane lines based on a comparison between the calculation result and a preset threshold.

[0021] Optionally, the calculation unit includes: acquiring first starting point coordinate information, first marker information, and first sequence number information of the object to be calculated from the first segmentation image, and acquiring second starting point coordinate information, second marker information, and second sequence number information of the object to be calculated from the second segmentation image; when it is determined through the first marker information and the second marker information that the category of the object to be calculated is lane line, and it is determined through the first sequence number information and the second sequence number information that the object to be calculated is located at the corresponding position in the first segmentation image and the second segmentation image, the first starting point coordinate information and the second starting point coordinate information are used to calculate the lane line length ratio of the object to be calculated, and the calculation result is obtained.

[0022] Optionally, the determining unit includes: determining the lane line classification result as a solid line when the calculation result is greater than a preset threshold; and determining the lane line classification result as a dashed line when the calculation result is less than or equal to the preset threshold.

[0023] According to one embodiment of the present invention, a non-volatile storage medium is also provided, characterized in that the storage medium stores a computer program, wherein the computer program is configured to execute the lane line detection method described in any of the preceding claims when running.

[0024] According to one embodiment of the present invention, a processor is also provided, characterized in that the processor is used to run a program, wherein the program is configured to execute the lane line detection method of any of the preceding claims when running.

[0025] According to one embodiment of the present invention, an electronic device is also provided, including a memory and a processor, characterized in that the memory stores a computer program, and the processor is configured to run the computer program to perform the lane detection method of any of the preceding claims.

[0026] In this embodiment of the invention, after acquiring an image including lane lines to be identified, a first neural network model is used to perform full-image segmentation analysis on the image to be identified, resulting in a first segmentation image that is not connected throughout the image; a second neural network model is used to perform lane line segmentation analysis on the image to be identified, resulting in a second segmentation image that is connected throughout the image. By using the aforementioned first and second segmentation images, the classification result of the lane lines is determined, thereby achieving the purpose of distinguishing between the real and virtual information of lane lines in the acquired image. This improves the accuracy of lane line detection and solves the technical problem that the advanced driver assistance system cannot distinguish between the real and virtual information of lane lines during vehicle operation, thus limiting the improvement of vehicle motion control capabilities. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0028] Figure 1 This is a flowchart of a lane line detection method according to one embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of a full-image semantic segmentation graph according to one embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of another lane line semantic segmentation map according to one embodiment of the present invention;

[0031] Figure 4 This is a structural block diagram of a lane line detection device according to one embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] According to an embodiment of the present invention, an embodiment of a lane line detection method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] This method embodiment can be executed in an electronic device or similar computing device that includes memory and a processor in a vehicle. Taking an electronic device running in a vehicle as an example, the vehicle's electronic device may include one or more processors (processors may include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), digital signal processing (DSP) chips, microprocessors (MCUs), programmable logic devices (FPGAs), neural network processors (NPUs), tensor processors (TPUs), artificial intelligence (AI) type processors, etc.) and memory for storing data. Optionally, the above-described automotive electronic device may also include transmission devices, input / output devices, and display devices for communication functions. Those skilled in the art will understand that the above structural description is merely illustrative and does not limit the structure of the vehicle's electronic device. For example, the vehicle's electronic device may also include more or fewer components than described above, or have a different configuration than described above.

[0036] The memory can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the information processing method in this embodiment of the invention. The processor executes various functional applications and data processing by running the computer program stored in the memory, thereby implementing the aforementioned information processing method. The memory may include high-speed random access memory (RAM) and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0037] The transmission device is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0038] The display device can be, for example, a touchscreen liquid crystal display (LCD) and a touch display (also referred to as a "touchscreen" or "touch display"). This LCD allows the user to interact with the user interface of the mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), which the user can interact with by touching a touch-sensitive surface with fingers and / or gestures. Optional human-computer interaction functions include: creating web pages, drawing, word processing, creating electronic documents, playing games, video conferencing, instant messaging, sending and receiving emails, a call interface, playing digital video, playing digital music, and / or web browsing, etc. Executable instructions for performing the above human-computer interaction functions are configured / stored in one or more processor-executable computer program products or readable storage media.

[0039] This embodiment provides a lane line detection method for an electronic device operating in the aforementioned vehicle. Figure 1 This is a flowchart of a lane line detection method according to one embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0040] Step S10: Obtain the image to be recognized, wherein the display content of the image to be recognized includes: lane lines to be recognized;

[0041] Step S12: Analyze the image to be recognized using the first neural network model to obtain the first segmentation map. The first neural network model is used to perform full image segmentation on the image to be recognized, and the first segmentation map is a segmentation map that is not connected to the whole image.

[0042] Step S14: Analyze the image to be recognized using the second neural network model to obtain the second segmentation map. The second neural network model is used to segment the lane lines, and the second segmentation map is a fully connected segmentation map.

[0043] Step S16: Determine the classification result of the lane lines based on the first segmentation map and the second segmentation map.

[0044] Through the above steps, after acquiring an image including the lane lines to be identified, a first neural network model is used to perform full-image segmentation analysis on the image to be identified, resulting in a first segmentation image that is not connected throughout the image. A second neural network model is then used to perform lane line segmentation analysis on the image to be identified, resulting in a second segmentation image that is connected throughout the image. By using the aforementioned first and second segmentation images, the classification result of the lane lines is determined, thereby achieving the goal of distinguishing between the real and virtual information of lane lines in the acquired image. This improves the accuracy of lane line detection and solves the technical problem that the advanced driver assistance system cannot distinguish between the real and virtual information of lane lines during vehicle operation, which limits the improvement of vehicle motion control capabilities.

[0045] Optionally, in step S12, the first neural network model is trained using multiple sets of image data through machine learning. Each set of image data includes: the original image and the categories of the full-image semantic segmentation labeled in the original image.

[0046] The aforementioned first neural network model can be a full-image segmentation model, used to perform full-image segmentation on the image to be recognized. The first segmentation map obtained by full-image segmentation is a segmentation map that is not connected in the full image. The generation process of the full-image segmentation model includes: learning and training the original image of the vehicle through the machine, and at the same time, learning and training the semantic segmentation categories labeled in the aforementioned original image through the machine.

[0047] Those skilled in the art will understand that full-image semantic segmentation refers to dividing a complex and irregular image into regions based on the attributes of objects, and labeling each region with a corresponding semantic label to help train an image recognition model. Full-image semantic segmentation can classify different categories of things, such as people, cars, and trees.

[0048] Figure 2 This is a schematic diagram of a full-image semantic segmentation graph according to one embodiment of the present invention. Figure 2 The corresponding original image is the image P1 to be recognized, then Figure 2 The specific process of its generation can be summarized as follows:

[0049] Prepare 100 original images F1 to F100, and manually label the categories of the whole image semantic segmentation on these original images to obtain the first training dataset D1 containing the categories of the whole image semantic segmentation. Based on D1, train the first neural network used for lane line detection. The first neural network model can be a whole image segmentation model.

[0050] Obtain the image P1 to be identified, input P1 into the trained full-image segmentation model, and output the full-image semantic segmentation map of P1 after processing by the full-image segmentation model (i.e., Figure 2 ).

[0051] Optionally, in step S14, the second neural network model is trained using multiple sets of image data through machine learning. Each set of image data includes: the original image and the lane lines marked in the original image.

[0052] The aforementioned second neural network model can be a lane line segmentation model, used to segment lane lines, and the resulting second segmentation map is a fully connected segmentation map; the generation process of this lane line segmentation model includes: learning and training the original image of the vehicle through the machine, and at the same time, learning and training the lane lines marked in the aforementioned original image through the machine.

[0053] Figure 3 This is a schematic diagram of another lane line semantic segmentation map according to one embodiment of the present invention. Figure 3 The corresponding original image is the image P1 to be recognized, then Figure 3 The specific process of its generation can be summarized as follows:

[0054] The lane lines of the aforementioned 100 original images F1 to F100 are manually annotated to obtain the second training dataset D2 containing lane lines. The second neural network used for lane line detection is trained based on D2. The second neural network model can be a lane line segmentation model.

[0055] Obtain the image P1 to be identified, input P1 into the lane line segmentation model that has been trained above, and output the lane line semantic segmentation map of P1 after processing by the lane line segmentation model (i.e., Figure 3 ).

[0056] Optionally, in step S16, determining the lane line classification result based on the first segmentation map and the second segmentation map may include the following execution steps:

[0057] Step S161: Project the first segmentation image and the second segmentation image onto the same target coordinate system;

[0058] Step S162: Under the target coordinate system, calculate the ratio of lane line lengths at corresponding positions in the first segmentation map and the second segmentation map to obtain the calculation result;

[0059] Step S163: Based on the comparison between the calculation results and the preset threshold, determine the classification result of the lane lines.

[0060] The first segmentation map is a segmentation map that is not connected to the whole map, and the second segmentation map is a segmentation map that is connected to the whole map. The feature information of the first segmentation map and the second segmentation map (including the lane line length at each position of the whole map) are projected into the same target coordinate system, and the ratio of the lane line length of the first segmentation map and the second segmentation map at the corresponding position is calculated based on the feature information. Finally, the classification result of the lane line is determined by comparing the aforementioned calculation result with the preset threshold.

[0061] In one embodiment of the present invention, it is still as follows Figure 2 , Figure 3 .in, Figure 2 The corresponding feature information includes: the lane line positions from left to right in the figure are W12, W12 and W13, the corresponding position coordinate information sets are x11, x12 and x13, and the corresponding lane line length information sets are L11, L12 and L13. Figure 3 The corresponding feature information includes: the positions of the lane lines from left to right in the image, respectively... Figure 2 W12, W12, and W13 correspond to W21, W22, and W23, respectively, with corresponding position coordinate information sets of x21, x22, and x23, and corresponding lane length information sets of L21, L22, and L23, respectively. Then, based on... Figure 2 and Figure 3 The specific implementation of determining the lane line classification results is as follows:

[0062] Select a target coordinate system for data projection, and then... Figure 2 and Figure 3 All feature information is projected onto the target coordinate system. The result is: Figure 2 The lane line positions W12′, W12′ and W13′; and their corresponding position coordinate information sets x11′, x12′ and x13′, and their corresponding lane line length information sets L11′, L12′ and L13′. Figure 3 The lane line positions W21′, W22′ and W23′, and their corresponding position coordinate information sets x21′, x22′ and x23′, and the corresponding lane line length information sets L21′, L22′ and L23′.

[0063] calculate Figure 2 and Figure 3The length ratios of lane lines at corresponding positions are as follows: the length ratio of the leftmost lane line is a1 = L21′ / L11′; the length ratio of the middle lane line is a2 = L22′ / L12′; and the length ratio of the rightmost lane line is a3 = L23′ / L13′.

[0064] A threshold Z is preset to determine the dashed or solid information of lane lines. Based on the relationship between the aforementioned calculation results a1, a2, a3 and Z, the classification result of the lane lines (i.e., whether they are dashed or solid lines) can be obtained.

[0065] Optionally, in step S162, calculating the ratio of lane line lengths at corresponding positions in the first segmentation map and the second segmentation map, and obtaining the calculation result may include the following steps:

[0066] Step S1621: Obtain the first starting point coordinate information, the first flag information, and the first sequence number information of the object to be calculated from the first segmentation image, and obtain the second starting point coordinate information, the second flag information, and the second sequence number information of the object to be calculated from the second segmentation image;

[0067] Step S1622: When it is determined through the first and second marker information that the category of the object to be calculated is lane line, and it is determined through the first and second sequence information that the object to be calculated is located at the corresponding position in the first and second segmentation maps, the lane line length ratio of the object to be calculated is calculated using the first and second starting point coordinate information to obtain the calculation result.

[0068] The aforementioned first and second marker information can be used to determine whether the object to be calculated is a lane line. The aforementioned first and second sequence number information can be used to determine the sequence number of the lane lines identified in the object to be calculated. The aforementioned first and second starting point coordinate information can be used to determine the endpoint coordinates of the lane lines identified in the object to be calculated. When calculating the aforementioned lane line length ratio, the aforementioned information is obtained from the first and second segmentation maps, and then two judgments are made: the first and second marker information determine that the category of the object to be calculated is lane lines; the first and second sequence number information determine that the object to be calculated is located at the corresponding position in the first and second segmentation maps. After both judgments pass, the lane line length ratio of the object to be calculated is calculated using the corresponding first and second starting point coordinate information to obtain the calculation result.

[0069] In one embodiment of the present invention, it is still as follows Figure 2 and Figure 3 As shown, Figure 2 The first flag information corresponding to the image P1 to be identified, the first sequence number information, and the first starting point coordinate information; Figure 3The second flag information, the second sequence number information, and the second starting point coordinate information corresponding to the image P1 to be identified.

[0070] Depend on Figure 2 As can be seen, only the first marker information of 3 objects to be calculated in the figure is marked as "lane line", corresponding to the leftmost, middle and rightmost lane lines in the figure. The first sequence information of these 3 objects to be calculated is 001, 002 and 003 respectively to confirm the corresponding position. The lane line lengths of these 3 objects to be calculated are 10.6, 11.0 and 10.5 respectively, based on the starting point coordinate information of these 3 objects to be calculated.

[0071] Depend on Figure 3 As can be seen, only the first marker information of 3 objects to be calculated in the figure is marked as "lane line", corresponding to the leftmost, middle and rightmost lane lines in the figure. The first sequence information of these 3 objects to be calculated is 001, 002 and 003 respectively, which are used to confirm the corresponding position. The corresponding lane line lengths are calculated to be 10.8, 6.4 and 10.4 respectively from the starting point coordinate information of these 3 objects to be calculated.

[0072] When calculating the lane length ratio, from Figure 2 and Figure 3 The aforementioned information is obtained from the corresponding feature data, and then two judgments are made: the first and second marker information determine that the category of the object to be calculated is lane line; the first and second sequence number information determine that the object to be calculated is located at the corresponding position in the first and second segmentation maps. The judgment is based solely on the aforementioned... Figure 2 and Figure 3 The lane length ratios for the six objects to be calculated were determined. The calculated lane length ratios for the three positions (leftmost, middle, and rearmost) were 1.02, 0.58, and 0.99, respectively.

[0073] Specifically, by Figure 3 It is evident that the break in the middle lane line inevitably leads to... Figure 3 The middle lane line corresponds to multiple objects to be calculated, and each corresponds to a starting point coordinate. In this case, there are two methods to calculate the length ratio of the middle lane line: (1) When calculating the length of the middle lane line from the aforementioned multiple starting point coordinates, calculate the sum of the lane line lengths of the multiple objects to be calculated corresponding to the middle lane line, and calculate the length ratio of the middle lane line based on the sum of the lane line lengths obtained. (2) Obtain the lengths of multiple partial lane lines of the multiple objects to be calculated corresponding to the middle lane line, calculate the partial lane line length ratio based on each partial lane line length, and finally sum the partial lane line length ratios of all the middle lane lines obtained above to obtain the length ratio of the middle lane line.

[0074] Optionally, in step S163, determining the lane line classification result based on the comparison between the calculation result and the preset threshold may include the following steps:

[0075] Step S1631: When the calculation result is greater than the preset threshold, the lane line classification result is determined to be a solid line;

[0076] Step S1632: When the calculation result is less than or equal to the preset threshold, the lane line classification result is determined to be a dashed line.

[0077] The calculation result can be the ratio of the lane line length at a certain location in the first segmentation image to the lane line length at the corresponding location in the second segmentation image. The preset threshold can be a pre-set critical value used to determine the solid or dashed information of the lane line at that location. The basis for comparing the calculation result with the preset threshold to determine the solid or dashed information of the lane line is: when the calculation result is greater than the preset threshold, the lane line is classified as a solid line; when the calculation result is less than or equal to the preset threshold, the lane line is classified as a dashed line.

[0078] In one embodiment of the present invention, as in the example above, a preset threshold of 0.6 is set to determine the solid / dark information of lane lines. That is, when the calculated lane line length ratio is greater than the preset threshold, the lane line is classified as a solid line; when the calculated lane line length ratio is less than or equal to the preset threshold, the lane line is classified as a dashed line. Obviously, in the example above, the leftmost and rightmost lane lines are solid lines, and the middle lane line is a dashed line.

[0079] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0080] This embodiment also provides a lane line detection device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0081] Figure 4This is a structural block diagram of a lane line detection device according to one embodiment of the present invention, such as... Figure 4 As shown, the device includes: an acquisition module 41 for acquiring an image to be recognized, wherein the display content of the image to be recognized includes lane lines to be recognized; a first analysis module 42 for analyzing the image to be recognized using a first neural network model to obtain a first segmentation map, wherein the first neural network model is used to perform full-image segmentation of the image to be recognized, and the first segmentation map is a segmentation map that is not connected to the whole image; a second analysis module 43 for analyzing the image to be recognized using a second neural network model to obtain a second segmentation map, wherein the second neural network model is used to segment lane lines, and the second segmentation map is a segmentation map that is connected to the whole image; and a detection module 44 for determining the classification result of the lane lines based on the first segmentation map and the second segmentation map.

[0082] Optionally, the first neural network model in the first analysis module 42 is trained using multiple sets of image data through machine learning. Each set of image data includes: the original image and the categories of the full-image semantic segmentation labeled in the original image.

[0083] Optionally, the second neural network model in the second analysis module 43 is trained using multiple sets of image data through machine learning. Each set of image data includes: the original image and the lane lines marked in the original image.

[0084] Optionally, the detection module 44 includes: a projection unit 441 (not shown in the figure), used to project the first segmentation image and the second segmentation image onto the same target coordinate system; a calculation unit 442 (not shown in the figure), used to calculate the ratio of lane line lengths at corresponding positions in the first segmentation image and the second segmentation image in the target coordinate system, and obtain the calculation result; and a determination unit 443 (not shown in the figure), used to determine the classification result of the lane line based on the comparison result of the calculation result and a preset threshold.

[0085] Optionally, the calculation unit 442 includes: acquiring first starting point coordinate information, first marker information, and first sequence number information of the object to be calculated from the first segmentation map, and acquiring second starting point coordinate information, second marker information, and second sequence number information of the object to be calculated from the second segmentation map; when it is determined through the first marker information and the second marker information that the category of the object to be calculated is lane line, and it is determined through the first sequence number information and the second sequence number information that the object to be calculated is located at the corresponding position in the first segmentation map and the second segmentation map, the first starting point coordinate information and the second starting point coordinate information are used to calculate the lane line length ratio of the object to be calculated, and the calculation result is obtained.

[0086] Optionally, the determining unit 443 includes: determining the classification result of the lane line as a solid line when the calculation result is greater than a preset threshold; and determining the classification result of the lane line as a dashed line when the calculation result is less than or equal to the preset threshold.

[0087] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0088] Embodiments of the present invention also provide a storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0089] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0090] Step S1: Obtain the image to be recognized, wherein the display content of the image to be recognized includes: lane lines to be recognized;

[0091] Step S2: Analyze the image to be recognized using the first neural network model to obtain the first segmentation map. The first neural network model is used to perform full image segmentation on the image to be recognized, and the first segmentation map is a segmentation map that is not connected to the whole image.

[0092] Step S3: Analyze the image to be recognized using the second neural network model to obtain the second segmentation map. The second neural network model is used to segment the lane lines, and the second segmentation map is a fully connected segmentation map.

[0093] Step S4: Determine the classification result of the lane lines based on the first segmentation map and the second segmentation map.

[0094] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0095] Embodiments of the present invention also provide a processor configured to run a computer program to perform the steps in any of the above method embodiments.

[0096] Step S1: Obtain the image to be recognized, wherein the display content of the image to be recognized includes: lane lines to be recognized;

[0097] Step S2: Analyze the image to be recognized using the first neural network model to obtain the first segmentation map. The first neural network model is used to perform full image segmentation on the image to be recognized, and the first segmentation map is a segmentation map that is not connected to the whole image.

[0098] Step S3: Analyze the image to be recognized using the second neural network model to obtain the second segmentation map. The second neural network model is used to segment the lane lines, and the second segmentation map is a fully connected segmentation map.

[0099] Step S4: Determine the classification result of the lane lines based on the first segmentation map and the second segmentation map.

[0100] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0101] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0102] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0107] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A lane line detection method, characterized in that, include: Acquire an image to be identified, wherein the display content of the image to be identified includes: lane lines to be identified; The first neural network model is used to analyze the image to be identified to obtain a first segmentation map. The first neural network model is trained by machine learning using multiple sets of image data. Each set of image data includes: the original image and the categories of the whole-image semantic segmentation labeled in the original image. The first neural network model is used to perform whole-image segmentation on the image to be identified. The first segmentation map is a whole-image disconnected segmentation map. The whole-image segmentation refers to dividing the image to be identified into regions according to the attributes of objects and labeling each region with a corresponding semantic label. The image to be identified is analyzed using a second neural network model to obtain a second segmentation map, wherein the second neural network model is used to segment the lane lines, and the second segmentation map is a fully connected segmentation map; Project the feature information of the first segmentation image and the second segmentation image onto the same target coordinate system; In the target coordinate system, the first starting point coordinate information, the first marker information, and the first sequence number information of the object to be calculated are obtained from the first segmentation map, and the second starting point coordinate information, the second marker information, and the second sequence number information of the object to be calculated are obtained from the second segmentation map. The first sequence number information and the second sequence number information are the sequence numbers of the lane lines identified in the object to be calculated, and the first starting point coordinate information and the second starting point coordinate information are the endpoint coordinates of the lane lines identified in the object to be calculated. When it is determined through the first flag information and the second flag information that the category of the object to be calculated is the lane line, and it is determined through the first sequence information and the second sequence information that the object to be calculated is located at the corresponding position in the first segmentation map and the second segmentation map, the lane line length ratio of the object to be calculated is calculated using the first starting point coordinate information and the second starting point coordinate information to obtain the calculation result; Based on the comparison between the calculation results and the preset threshold, the classification result of the lane lines is determined.

2. The lane line detection method according to claim 1, characterized in that, The second neural network model is trained using multiple sets of image data through machine learning. Each set of image data includes: the original image and the lane lines marked in the original image.

3. The lane line detection method according to claim 1, characterized in that, Based on the comparison between the calculation result and the preset threshold, the classification result of the lane line is determined as follows: When the calculation result is greater than the preset threshold, the lane line is classified as a solid line. When the calculation result is less than or equal to the preset threshold, the lane line is classified as a dashed line.

4. A lane line detection device, characterized in that, include: An acquisition module is used to acquire an image to be recognized, wherein the display content of the image to be recognized includes: lane lines to be recognized; The first analysis module is used to analyze the image to be identified using a first neural network model to obtain a first segmentation map. The first neural network model is trained using multiple sets of image data through machine learning. Each set of image data includes: the original image and the categories of the whole-image semantic segmentation labeled in the original image. The first neural network model is used to perform whole-image segmentation on the image to be identified. The first segmentation map is a whole-image disconnected segmentation map. The whole-image segmentation refers to dividing the image to be identified into regions according to the attributes of objects and labeling each region with a corresponding semantic label. The second analysis module is used to analyze the image to be identified using a second neural network model to obtain a second segmentation map, wherein the second neural network model is used to segment the lane lines, and the second segmentation map is a fully connected segmentation map; The detection module is used to project the feature information of the first segmentation image and the second segmentation image onto the same target coordinate system; under the target coordinate system, it obtains the first starting point coordinate information, the first marker information, and the first sequence number information of the object to be calculated from the first segmentation image, and the second starting point coordinate information, the second marker information, and the second sequence number information of the object to be calculated from the second segmentation image, wherein the first sequence number information and the second sequence number information are the sequence numbers of the lane lines already identified in the object to be calculated, and the first starting point coordinate information and the second starting point coordinate information are the endpoint coordinates of the lane lines already identified in the object to be calculated; when it is determined by the first marker information and the second marker information that the category of the object to be calculated is the lane line, and it is determined by the first sequence number information and the second sequence number information that the object to be calculated is located at the corresponding position in the first segmentation image and the second segmentation image, it calculates the lane line length ratio of the object to be calculated using the first starting point coordinate information and the second starting point coordinate information to obtain the calculation result; based on the comparison result of the calculation result and the preset threshold, it determines the classification result of the lane line.

5. A non-volatile storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the lane line detection method according to any one of claims 1 to 3 when it is run.

6. A processor, characterized in that, The processor is used to run a program, wherein the program is configured to execute the lane line detection method according to any one of claims 1 to 3 at runtime.

7. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the lane detection method according to any one of claims 1 to 3.

Images