A method and apparatus for detecting lane lines

Abstract

This specification discloses a lane line detection method and apparatus, which extracts lane lines and reference lines from a target image, maps them into a preset virtual space, determines the lane line topology relationship between each lane line mapped into the virtual space based on the reference lines mapped into the virtual space, and fuses the lane lines mapped into the virtual space based on the lane line topology relationship, and determines the lane lines not detected from the target image by the fused lane lines.

Description

Technical Field

[0001] This specification relates to the field of autonomous driving technology, and in particular to a lane line detection method and device. Background Technology

[0002] In an autonomous driving environment, it is necessary to build a map of the unmanned equipment in advance. This map needs to include environmental information, such as lane markings, to enable the unmanned equipment to make reasonable driving decisions. Summary of the Invention

[0003] This specification provides a lane line detection method and apparatus to partially solve the aforementioned problems existing in the prior art.

[0004] The following technical solution is adopted in this specification:

[0005] This specification provides a lane line detection method, including:

[0006] The lane lines and reference lines extracted from the target image are mapped into a preset virtual space, and the lane line topology relationship between each lane line mapped into the virtual space is determined based on the reference lines mapped into the virtual space.

[0007] Based on the lane line topology, the lane lines mapped into the virtual space are fused, and the fused lane lines are determined as the lane lines detected from the target image.

[0008] Optionally, based on the reference lines mapped into the virtual space, the lane line topology relationship between the lane lines mapped into the virtual space is determined, specifically including:

[0009] Based on the reference lines and lane lines mapped into the virtual space, determine the positional relationship between each lane line and the reference line mapped into the virtual space.

[0010] Based on the positional relationships, determine the lane line topology relationships between the lane lines mapped to the virtual space.

[0011] Optionally, based on the lane line topology, the lane lines mapped into the virtual space are merged to obtain merged lane lines, specifically including:

[0012] Multiple lane lines in the virtual space that conform to the lane line topology are considered as the same group of lane lines;

[0013] For each set of lane lines, merge the set of lane lines into one lane line to obtain the merged lane line.

[0014] Optionally, based on the lane line topology, the lane lines mapped into the virtual space are merged to obtain merged lane lines, specifically including:

[0015] Determine the confidence level of each lane line that conforms to the lane line topology;

[0016] Based on the confidence level, lane lines that conform to the lane line topology are merged in the virtual space to obtain merged lane lines.

[0017] Optionally, determining the confidence level corresponding to each lane line that conforms to the lane line topology relationship specifically includes:

[0018] For each lane line that conforms to the lane line topology, determine the pose on which the acquisition device acquires a target image containing the image of that lane line;

[0019] Based on the pose, determine the distance between the acquisition device and the lane line when the acquisition device acquires the target image containing the lane line;

[0020] Based on the distance, the confidence level corresponding to the lane line is determined, wherein the confidence level and the distance are negatively correlated.

[0021] Optionally, the virtual space includes: a three-dimensional road space;

[0022] The lane lines and reference lines extracted from the target image are mapped onto a preset virtual space, specifically including:

[0023] Based on the feature points extracted from the target image, lane lines and reference lines are fused into the target image;

[0024] The lane lines and reference lines fused from the target image are mapped onto a preset three-dimensional road space.

[0025] Optionally, the lane lines and reference lines extracted from the target image are mapped into a preset virtual space, specifically including:

[0026] Based on the depth value of each pixel in the target image and the pose on which the acquisition device acquires the target image, the lane lines and reference lines extracted from the target image are mapped into a preset virtual space.

[0027] Optionally, the method further includes:

[0028] A map is constructed based on the merged lane lines.

[0029] Optionally, the reference line includes at least one of: the road centerline and the curb line formed by the merging of the curb.

[0030] This specification provides a lane line detection device, comprising:

[0031] The mapping module is used to map the lane lines and reference lines extracted from the target image into a preset virtual space, and to determine the lane line topology relationship between each lane line mapped into the virtual space based on the reference lines mapped into the virtual space.

[0032] The detection module is used to fuse the lane lines mapped into the virtual space according to the lane line topology, and determine the fused lane lines as the lane lines detected from the target image.

[0033] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the lane detection method described above.

[0034] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the lane line detection method described above.

[0035] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0036] The lane line detection method provided in this specification maps lane lines and reference lines extracted from the target image into a preset virtual space, determines the lane line topology relationship between each lane line mapped into the virtual space based on the reference lines mapped into the virtual space, and fuses the lane lines mapped into the virtual space based on the lane line topology relationship, and determines the lane lines not detected from the target image by the fused lane lines.

[0037] As can be seen from the above method, the lane detection method eliminates the need for expensive sensors such as LiDAR to collect environmental information. It only requires image data from a camera to construct a map for unmanned vehicles, thus reducing map construction costs. Furthermore, the lane detection method provided in this specification fuses lane lines based on their topological relationships, grouping the same lane line into a single lane line. Compared to existing technologies, this effectively improves the accuracy of the fused lane lines, thereby ensuring the precision of the constructed map. Attached Figure Description

[0038] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0039] Figure 1 This is a flowchart illustrating a lane line detection method provided in this specification;

[0040] Figure 2 This specification provides a schematic diagram of lane markings.

[0041] Figure 3 This is a schematic diagram of the reference lines provided in this instruction manual;

[0042] Figures 4A-4B This is a schematic diagram illustrating the method for determining the topological relationships between lane lines provided in this specification.

[0043] Figure 5 This is a schematic diagram of a lane line detection device provided in this specification;

[0044] Figure 6 This specification provides a corresponding Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0046] In the current field of autonomous driving, the main method for constructing driving maps for unmanned vehicles is to collect high-precision sensor data (such as point cloud data) using acquisition devices equipped with high-precision sensors like LiDAR, and then construct a map based on this high-precision sensor data. While this method can obtain highly accurate maps, the cost of high-precision sensors like LiDAR is relatively high.

[0047] In addition, another way to construct a map for unmanned equipment is to use a data acquisition device equipped with a camera to collect image information, extract relevant map information, such as lane line information, from multiple images. Based on the position of the camera set on the data acquisition device, a depth map of the equipment's driving environment is generated, and multiple depth maps are projected into a three-dimensional space for fusion to generate the map used by the unmanned equipment.

[0048] While this method allows for the low-cost creation of maps for unmanned equipment, it ultimately struggles to group lane lines into a single lane, resulting in lower map accuracy.

[0049] Therefore, how to reduce the cost of map construction while ensuring the accuracy of the constructed map is an urgent problem to be solved.

[0050] To address the aforementioned problems, this specification provides a method for lane line detection. The technical solutions provided by the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0051] Figure 1 This is a flowchart illustrating a lane line detection method provided in this specification, including the following steps:

[0052] S101: The lane lines and reference lines extracted from the target image are mapped into a preset virtual space, and the lane line topology relationship between each lane line mapped into the virtual space is determined based on the reference lines mapped into the virtual space.

[0053] In this specification, the entity implementing the lane line detection method can be a desktop computer, laptop computer, or other terminal device; a server or other equipment; or an unmanned device, such as an unmanned vehicle, drone, or autonomous robot. The map constructed by the lane line detection method provided in this specification allows unmanned devices to perform delivery tasks in the delivery field, such as in scenarios involving express delivery, logistics, and food delivery. For ease of description, the lane line detection method provided in this specification will be described in detail below, focusing solely on the terminal device as the implementing entity.

[0054] Currently, in the field of autonomous driving, the main method for constructing maps for unmanned vehicles involves using data acquisition devices equipped with high-precision sensors (such as unmanned vehicles or manned control devices) to collect environmental information about the map area where lane lines need to be constructed, and then building the map based on this collected environmental information. However, due to the high cost of high-precision sensors, lane detection methods relying on high-precision sensors are difficult to implement for constructing large-scale maps.

[0055] In addition, another way to build maps for autonomous driving equipment is to use images of the map area where lane lines need to be built, collected by cameras on the equipment, extract feature points from them, and build the map based on the collected feature points.

[0056] While this method enables low-cost map construction, it ultimately struggles to cluster multiple lane lines corresponding to a single lane line in real space into a single lane line, resulting in a low-accuracy map.

[0057] To address the aforementioned technical problems, this specification provides a lane line detection method. First, the terminal device determines at least one map region from which lane lines need to be constructed, as the target region. Here, lane lines are lines used to regulate vehicle trajectories. For example... Figure 2 This is a lane marking diagram provided in this manual. Lanes 1 to 3 are three lane markings, and vehicles can travel between lane 1 and lane 2, or between lane 2 and lane 3.

[0058] The target area mentioned above can be the entire map to be constructed, or it can be a part of the map to be constructed. This specification does not specifically limit the type of target area.

[0059] It should be noted that the image data mentioned in this manual refers to two-dimensional image data acquired by image acquisition devices such as cameras and camcorders, rather than three-dimensional point cloud images acquired by lidar.

[0060] After determining the target area where lane lines need to be constructed, the terminal device uses image data collected from the actual geographical area corresponding to the target area by a pre-acquired acquisition device as the target image. The acquisition device can be an image acquisition device such as a camera or webcam carried by unmanned equipment, manned control equipment, or carried by personnel.

[0061] In this specification, the target image can be image data containing the iconic structures of the target area or image data whose clarity meets a preset clarity requirement. For example, if the image data contains lane lines and / or curbs of the target area, then that image data is used as the target image. As another example, if the image data has high clarity and lane lines can be identified from it, then that image data can be used as the target image.

[0062] It should be noted that when the acquisition device collects image data, it can record the location information on which the acquisition of this image data was based. This information can then be stored in a database, corresponding to the image data and the location information used to acquire it. When constructing a map, the image data used to construct the map of the target area can be retrieved based on the location information recorded in the database.

[0063] After acquiring the target image, the terminal device can extract lane lines and reference lines within the target area for each target image. The reference lines can be at least one of the following: the road centerline and a curb line blended from the curb. For example... Figure 3 This is a schematic diagram of the reference lines provided in this manual. Lines 2 and 4 are lane lines, lines 1 and 5 are roadside lines fused from roadside information collected from the target image, and line 3 is the road centerline, a marking line used to distinguish the driving space of vehicles traveling in both directions.

[0064] One method for a terminal device to extract lane lines and reference lines within a target area is to extract feature points from each target image and determine the lane lines and / or reference lines based on these extracted feature points. Specifically, the terminal device can input the target image into a preset image recognition model. The image recognition model identifies the lane lines or reference lines contained in the target image by extracting the feature points. Alternatively, the terminal device can also identify lane lines or reference lines in the target image using other methods. For example, after extracting the feature points from the target image, the terminal device can fuse the feature points based on their positions in the target image to obtain lines, and then identify the lane lines or reference lines in the target image based on the positions of these lines.

[0065] It should be noted that image recognition models can identify lane lines in each target image through semantic analysis. After identifying the lane lines in each target image, a pre-defined tracking algorithm (such as the Kalman filter algorithm) can be used to track the lane lines. Furthermore, it is important to emphasize that the lane line tracking mentioned here only associates lane lines that may have topological relationships between consecutive frames, rather than directly determining the topological relationships between lane lines.

[0066] Specifically, for the road centerline, if two road directional signs in opposite directions are identified from a line in the target image, then this line can be determined as the road centerline. For the curb line, the curb identified from the target image can be projected onto the plane where the ground is located in the target image to determine the corresponding curb line.

[0067] After the terminal device acquires the lane lines and reference lines within the target area extracted from each target image, it maps these lane lines and reference lines into a preset virtual space. This preset virtual space can be a three-dimensional road space. That is, for each target image, the terminal device can fuse lane lines and reference lines from the target image based on the feature points extracted from it. Then, the terminal device can map the fused lane lines and reference lines from each target image into the preset three-dimensional road space.

[0068] After the terminal device maps each lane line and reference line to a preset virtual space, the topological relationship between the lane lines mapped to the virtual space can be determined based on the reference lines in the virtual space. Specifically, the topological relationship between lane lines can be determined if each lane line extracted from each target image corresponds to the same lane line in real space.

[0069] In practical applications, different lane lines in the same lane often correspond to the same road center line or the road edge line formed by merging the road edge. Based on this, in this specification, the terminal device can determine the topological relationship between lane lines according to the positional relationship between each lane line and the reference line in the virtual space mapped to the lane.

[0070] like Figures 4A-4B This is a schematic diagram illustrating the method for determining the topological relationships between lane lines provided in this specification. Figure 4A and Figure 4B To capture images of different targets on the same road segment, Figure 4A In the diagram, lines 2, 3, 5, and 6 are lane markings; lines 1 and 7 are curb lines fused from curb information collected from target images; and line 4 is the road centerline. Figure 4B In the diagram, lines 9, 10, 12, and 13 are lane lines, lines 8 and 14 are roadside lines fused from roadside information collected from target images, and line 11 is the road centerline.

[0071] In determining Figure 4A and Figure 4B When the data collected is for the same road segment, it can be determined that for lane 2, the left side is the curb line and the right side has a lane line, meaning there is a lane line sandwiched between the curb line and the center line. For lane 3, the right side is the center line and the left side has a lane line, meaning there is a lane line sandwiched between the curb line and the left side. For lane 5, the left side is the center line and the right side has a lane line, meaning there is a lane line sandwiched between the curb line and the right side. For lane 6, the right side is the curb line and the left side has a lane line, meaning there is a lane line sandwiched between the curb line and the center line.

[0072] Similarly, for Line 9, the left side is the curb line, and the right side has a lane line, meaning there's a lane line sandwiched between the curb line and the center line. For Line 10, the right side is the center line, and the left side has a lane line, meaning there's a lane line sandwiched between the curb line and the left side. For Line 12, the left side is the center line, and the right side has a lane line, meaning there's a lane line sandwiched between the curb line and the right side. For Line 13, the right side is the curb line, and the left side has a lane line, meaning there's a lane line sandwiched between the curb line and the center line.

[0073] Based on the above conditions, we can conclude that Line 2 and Line 9 share the same lane, Line 3 and Line 10 share the same lane, Line 5 and Line 12 share the same lane, and Line 6 and Line 13 share the same lane. Therefore, Line 2 and Line 9 have a topological relationship, Line 3 and Line 10 have a topological relationship, Line 5 and Line 12 have a topological relationship, and Line 6 and Line 13 have a topological relationship.

[0074] Of course, the lane line topology mentioned above can be used not only to represent the relationship between lane lines in image data collected from different viewpoints at the same location (such as images collected by a binocular camera), but also to represent the relationship between lane lines in image data collected from different locations. For example, when a data acquisition device travels along a road, it acquires two image data at time t and time t+1. These two image data correspond to different acquisition locations, but the lane lines contained in these two image data actually originate from the same lane lines on that road. Therefore, in this specification, the terminal device can also determine the topological relationship between lane lines contained in different target images based on the acquisition location and acquisition time of the target image.

[0075] In the specific mapping process, the terminal device can first determine the depth value of each pixel in the target image (mainly the depth values ​​of lane lines and reference lines), as well as the pose on which the acquisition device acquired the target image. Then, based on the determined depth values ​​and pose, the pixels of lane lines and reference lines in the target image are mapped into the virtual space. One method for determining the depth value of each pixel in the target image is to use the DEEPV2D algorithm.

[0076] It should be noted that, in this specification, in addition to mapping lane lines or reference lines identified from the target image to a three-dimensional virtual space, the terminal device can also map the identified lane lines or reference lines to a two-dimensional virtual space, such as mapping the identified lane lines or reference lines to a two-dimensional map space with a top-down view.

[0077] S102: Based on the lane line topology, the lane lines mapped into the virtual space are fused, and the fused lane lines are determined as the lane lines detected from the target image.

[0078] After acquiring the topological relationships between lane lines, the terminal device will group multiple lane lines that conform to the lane line topology in the virtual space as lane lines in the same group. For each group of lane lines, the terminal device can determine the final merged lane line from that group of lane lines according to a preset lane line merging method.

[0079] The lane line fusion method can be as follows: in a preset virtual space, for each group of lane lines, the terminal device can determine the center position of the group of lane lines based on the positions of the lane lines at both ends of the group of lane lines, and take the lane line closest to the center position of the group of lane lines as the final merged lane line.

[0080] The terminal device can also fuse lane lines based on the confidence levels corresponding to different lane lines. Specifically, for each lane line conforming to the lane line topology, the terminal device can determine the pose on which the acquisition device collects image data containing that lane line. Based on the pose of the acquisition device, the terminal device can determine the distance between the acquisition device and the lane line when collecting image data containing that lane line. Based on the distance between the acquisition device and the lane line, the terminal device determines the confidence level corresponding to that lane line. Notably, the confidence level and the distance between the acquisition device and the lane line are negatively correlated. That is, the closer the acquisition device is to the lane line when acquiring the target image, the higher the confidence level of the lane line contained in the target image.

[0081] The terminal device can select the lane line with the highest confidence level as the final merged lane line. Alternatively, the terminal device can merge a group of lane lines based on the confidence levels of different lane lines to obtain the final merged lane line. That is, the terminal device can use the direction perpendicular to the group of lane lines as the cutting direction to cut the group of lane lines into multiple segments, and then select the lane line with the highest confidence level for each segment as the corresponding lane line. Finally, using the merged lane line as a straight line as a constraint, the corresponding lane lines of each segment are merged to obtain the merged lane line.

[0082] Based on the lane line fusion method described above, the terminal device can merge multiple lane lines into one lane line, thereby solving to some extent the problem that when building a map, the same lane line cannot be clustered into one lane line, or the curvature of the lane line obtained by final clustering is significantly different from the actual curvature.

[0083] The terminal device can construct a map of the target area based on the fused lane lines acquired in the three-dimensional road space. As can be seen from the lane line detection method provided in this specification, the terminal device does not need to use expensive sensors such as LiDAR to collect environmental information; it only needs to construct the driving map of the unmanned equipment using image data collected by cameras and other acquisition devices, thus reducing the cost of map construction. Furthermore, the lane line fusion method provided in this specification can merge lane lines in the same group into a single lane line, which effectively improves the accuracy of the fused lane lines compared to existing technologies, thereby ensuring the accuracy of the final constructed map.

[0084] Figure 5 A schematic diagram of a lane line detection device provided in this specification includes:

[0085] The mapping module 501 is used to map the lane lines and reference lines extracted from the target image into a preset virtual space, and to determine the lane line topology relationship between each lane line mapped into the virtual space based on the reference lines mapped into the virtual space.

[0086] The detection module 502 is used to fuse the lane lines mapped to the virtual space according to the lane line topology relationship, and determine the fused lane lines as the lane lines detected from the target image.

[0087] Optionally, the mapping module 501 is specifically used to: determine the positional relationship between each lane line mapped to the virtual space and the reference line based on the reference line and lane lines mapped to the virtual space; and determine the lane line topology relationship between each lane line mapped to the virtual space based on the positional relationship.

[0088] Optionally, the mapping module 501 is specifically used to take multiple lane lines in the virtual space that conform to the lane line topology as the same group of lane lines; and for each group of lane lines, merge the group of lane lines into one lane line to obtain the merged lane line.

[0089] Optionally, the mapping module 501 is specifically used to determine the confidence level of each lane line that conforms to the lane line topology; and to merge the lane lines that conform to the lane line topology in the virtual space according to the confidence level to obtain the merged lane lines.

[0090] Optionally, the detection module 502 is specifically configured to: for each lane line conforming to the lane line topology, determine the pose on which the acquisition device acquires the target image containing the lane line; determine the distance between the acquisition device and the lane line when acquiring the target image containing the lane line based on the pose; and determine the confidence level corresponding to the lane line based on the distance, wherein the confidence level and the distance are negatively correlated.

[0091] Optionally, the virtual space includes: a three-dimensional road space;

[0092] The detection module 502 is specifically used to fuse lane lines and reference lines in the target image based on feature points extracted from the target image; and to map the lane lines and reference lines fused in the target image into a preset three-dimensional road space.

[0093] Optionally, the mapping module 501 is specifically used to map the lane lines and reference lines extracted from the target image into a preset virtual space based on the depth value of each pixel in the target image and the pose on which the acquisition device acquires the target image.

[0094] Optionally, the detection module 502 is further configured to construct a map based on the fused lane lines.

[0095] Optionally, the reference line includes at least one of: the road centerline and the curb line formed by the merging of the curb.

[0096] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 A lane line detection method is provided.

[0097] This instruction manual also provides Figure 6 One of the corresponding Figure 1 A schematic diagram of the structure of an electronic device. (e.g.) Figure 6 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The lane line detection method described above. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0098] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0099] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0100] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0101] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0102] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0104] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0105] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0106] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0107] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0108] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0109] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0110] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0112] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0113] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A lane line detection method, characterized in that, include: Lane lines and reference lines extracted from the target image are mapped into a preset virtual space. Based on the reference lines mapped into the virtual space, the lane line topology relationship between the lane lines mapped into the virtual space is determined. If each lane line extracted from each target image corresponds to the same lane line in real space, then it is determined that these lane lines have a topological relationship. Based on the lane line topology relationship, the lane lines mapped into the virtual space are fused, and the fused lane lines are determined as the lane lines detected from the target image. Based on the reference lines mapped into the virtual space, determine the lane line topology relationship between each lane line mapped into the virtual space, specifically including: based on the reference lines and lane lines mapped into the virtual space, determine the positional relationship between each lane line and the reference lines mapped into the virtual space. Based on the positional relationships, determine the lane line topology relationships between the lane lines mapped to the virtual space.

2. The method as described in claim 1, characterized in that, Based on the lane line topology, the lane lines mapped into the virtual space are merged, specifically including: taking multiple lane lines in the virtual space that conform to the lane line topology as the same group of lane lines; and merging each group of lane lines into one lane line to obtain the merged lane line.

3. The method as described in claim 1 or 2, characterized in that, Based on the lane line topology, the lane lines mapped into the virtual space are fused, specifically including: determining the confidence level of each lane line that conforms to the lane line topology; and fusion of the lane lines that conform to the lane line topology in the virtual space based on the confidence level to obtain the fused lane lines.

4. The method as described in claim 3, characterized in that, Determining the confidence level of each lane line that conforms to the lane line topology relationship specifically includes: for each lane line that conforms to the lane line topology relationship, determining the pose on which the acquisition device acquires the target image containing the image of the lane line; determining the distance between the acquisition device and the lane line when acquiring the target image containing the image of the lane line based on the pose; and determining the confidence level corresponding to the lane line based on the distance, wherein the confidence level and the distance are negatively correlated.

5. The method as described in claim 1, characterized in that, The virtual space includes: a three-dimensional road space; mapping lane lines and reference lines extracted from the target image into the preset virtual space, specifically including: fusing lane lines and reference lines in the target image based on feature points extracted from the target image; and mapping the lane lines and reference lines fused in the target image into the preset three-dimensional road space.

6. The method as described in claim 1, characterized in that, The lane lines and reference lines extracted from the target image are mapped into a preset virtual space. Specifically, this includes mapping the lane lines and reference lines extracted from the target image into a preset virtual space based on the depth value of each pixel in the target image and the pose on which the acquisition device acquires the target image.

7. The method as described in claim 1, characterized in that, The method further includes: constructing a map based on the fused lane lines.

8. The method as described in claim 1, characterized in that, The reference line includes at least one of the following: the road centerline and the curb line formed by the merging of the curb.

9. A lane line detection device, characterized in that, include: The mapping module is used to map the lane lines and reference lines extracted from the target image into a preset virtual space, and to determine the lane line topology relationship between each lane line mapped into the virtual space based on the reference lines mapped into the virtual space. If each lane line extracted from each target image corresponds to the same lane line in the real space, then it is determined that there is a topological relationship between these lane lines. The detection module is used to fuse the lane lines mapped into the virtual space according to the lane line topology relationship, and determine the fused lane lines as the lane lines detected from the target image. Based on the reference lines mapped into the virtual space, determine the lane line topology relationship between each lane line mapped into the virtual space, specifically including: based on the reference lines and lane lines mapped into the virtual space, determine the positional relationship between each lane line and the reference lines mapped into the virtual space. Based on the positional relationships, determine the lane line topology relationships between the lane lines mapped to the virtual space.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 8.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 8.

Images