Method and apparatus for detecting lane lines

Abstract

The application provides a lane line detection method and device in the field of artificial intelligence. The method comprises: scanning the surrounding environment of a vehicle by using a laser radar to obtain lane line candidate reflection points and road edge information; establishing a road edge coordinate system based on the road edge information; extracting lane line reflection points from the lane line candidate reflection points based on the coordinates of the lane line candidate reflection points in the road edge coordinate system; and obtaining the lane line based on the lane line reflection points. According to the technical scheme, the road edge coordinate system is established according to the road edge information, and the lane line reflection points are extracted in the road edge coordinate system, so that the detection accuracy and stability of the lane line can be improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and more specifically, to a method and apparatus for detecting lane lines. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. Research in the field of AI includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and fundamental AI theories.

[0003] Autonomous driving is a mainstream application of artificial intelligence. Autonomous driving technology relies on the collaborative efforts of computer vision, radar, monitoring devices, and global positioning systems to enable motor vehicles to operate autonomously without human intervention. Autonomous vehicles use various computing systems to help transport passengers from one location to another. Some autonomous vehicles may require initial or continuous input from an operator (such as a navigator, driver, or passenger). Autonomous vehicles allow operators to switch from manual mode to autonomous driving mode or a mode in between. Because autonomous driving technology eliminates the need for human drivers, it theoretically can effectively avoid human driving errors, reduce traffic accidents, and improve highway transportation efficiency. Therefore, autonomous driving technology is receiving increasing attention.

[0004] In the field of autonomous driving, vehicle positioning is essential to ensure that the vehicle does not deviate from the road. Among various road markings, lane lines play a crucial role in ensuring the accuracy of vehicle positioning. Therefore, the accuracy of lane line detection directly affects whether the vehicle can be accurately positioned. Summary of the Invention

[0005] This application provides a lane line detection method and apparatus, which can improve the detection accuracy and stability of lane lines, thereby achieving precise vehicle positioning.

[0006] In a first aspect, a method for detecting lane lines is provided, comprising: scanning the surrounding environment of a vehicle using a lidar to obtain lane line candidate reflection points and road edge information; establishing a road edge coordinate system based on the road edge information; extracting lane line reflection points from the lane line candidate reflection points based on the coordinates of the lane line candidate reflection points in the road edge coordinate system; and obtaining lane lines based on the lane line reflection points.

[0007] Since there is a certain correlation between road edges and lane lines, using road edge information as prior information can effectively filter out noise in candidate lane line reflection points, improving the accuracy and stability of lane line detection. Furthermore, regardless of the actual shape of the lane lines, the arrangement of lane line reflection points in the road edge coordinate system is relatively regular; therefore, extracting lane line reflection points in the road edge coordinate system is also simpler.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, before extracting lane line reflection points from the lane line candidate reflection points based on the coordinates of the lane line candidate reflection points in the road edge coordinate system, the method further includes: filtering out reflection points from the lane line candidate reflection points whose distance from the road edge is greater than a threshold based on the road edge information.

[0009] Reflection points at road edges and lane line reflection points share similar characteristics, both exhibiting strong reflection intensity. Therefore, road edge reflection points are often included as noise in the candidate lane line reflection points. Before processing the candidate lane line reflection points using the road edge coordinate system, noise introduced by the road edge can be pre-filtered based on the distance relationship between the road edge and the candidate reflection points, thereby improving the efficiency of subsequent processing.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the road edge information includes information on two road edge lines, and the establishment of a road edge coordinate system based on the road edge information includes: using the longer of the two road edge lines as a reference line to establish the road edge coordinate system.

[0011] Longer edge lines are more reliable. Using longer edge lines as reference lines to establish a road edge coordinate system will also make the lane line detection results more reliable.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the road edge information includes information on two road edge lines, the two road edge lines being different, and the road edge coordinate system includes two coordinate systems established with the two road edge lines as reference lines respectively.

[0013] When the edge lines of two roads differ significantly, using two coordinate systems to screen candidate reflection points for lane lines makes it less likely to miss lane line reflection points, and the detection results for lane lines will be more accurate.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the step of extracting lane line reflection points from the lane line candidate reflection points based on the coordinates of the lane line candidate reflection points in the road edge coordinate system includes: determining the features of the lane line candidate reflection points based on the coordinates of the lane line candidate reflection points in the road edge coordinate system; and extracting the lane line reflection points from the lane line candidate reflection points based on the features of the lane line candidate reflection points.

[0015] The characteristics of lane line candidate reflection points may include one or more of the following: the orientation of the lane line candidate reflection points, the distance between the lane line candidate reflection points and the road edge, the spacing between the lane line candidate reflection points and each other, and whether the lane line candidate reflection points can form a straight line.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the road edge information includes information on two road edge lines, and the establishment of a road edge coordinate system based on the road edge information includes: establishing the road edge coordinate system using the center line of the two road edge lines as a reference line.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, determining the features of the lane line candidate reflection points based on their coordinates in the road edge coordinate system includes: performing feature detection using a Hough transform based on the coordinates of the lane line candidate reflection points in the road edge coordinate system to obtain a straight line; extracting the lane line reflection points from the lane line candidate reflection points based on their features includes: extracting reflection points from the lane line candidate reflection points whose distance from the straight line is less than a first threshold, and using these as the lane line reflection points.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, extracting lane line reflection points from the lane line candidate reflection points based on the coordinates of the lane line candidate reflection points in the road edge coordinate system includes: clustering the lane line candidate reflection points based on the coordinates of the lane line candidate reflection points in the road edge coordinate system; and determining the lane line reflection points based on the clustering results.

[0019] In a second aspect, a lane line detection device is provided, including a module for performing the detection method described in the first aspect.

[0020] Thirdly, a lane line detection device is provided, comprising: at least one processor; and a memory signal-connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, cause the at least one processor to perform the detection method described in the first aspect.

[0021] Fourthly, a chip is provided, the chip including a processor and a data interface, the processor reading instructions stored in a memory through the data interface and executing the detection method described in the first aspect.

[0022] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to perform the detection method described in the first aspect.

[0023] Fifthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the detection method described in the first aspect.

[0024] In a sixth aspect, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the detection method described in the first aspect.

[0025] In a seventh aspect, a vehicle is provided that includes the detection device described in the second or third aspect. Attached Figure Description

[0026] Figure 1 A functional block diagram of a vehicle provided in an embodiment of this application.

[0027] Figure 2 This is a schematic diagram of the structure of an autonomous driving system for a vehicle provided in an embodiment of this application.

[0028] Figure 3 for Figure 2 A schematic diagram of the specific structure of the environment perception module 220.

[0029] Figure 4 This is a flowchart illustrating the lane line detection method provided in an embodiment of this application.

[0030] Figure 5 for Figure 4 A flowchart illustrating one possible implementation of step S410 in the above steps.

[0031] Figure 6 This is a schematic diagram showing the distribution of candidate reflection points of the lane line in the vehicle coordinate system.

[0032] Figure 7 This is a schematic diagram showing the distribution of candidate reflection points of lane lines in the road edge coordinate system.

[0033] Figure 8 for Figure 4 A flowchart illustrating one possible implementation of step S430 in the above steps.

[0034] Figure 9 for Figure 4 A flowchart illustrating another possible implementation of step S430 in the above steps.

[0035] Figure 10 for Figure 4 A flowchart illustrating another possible implementation of step S430 in the above steps.

[0036] Figure 11 This is a schematic diagram of the structure of a lane line detection device provided in an embodiment of this application.

[0037] Figure 12 This is a schematic diagram of the structure of a lane line detection device provided in another embodiment of this application. Detailed Implementation

[0038] The lane line detection method provided in this application can be applied to vehicles with autonomous driving functions. The following is a combination of... Figure 1 A detailed description of the specific structure of vehicles with autonomous driving capabilities is provided.

[0039] Figure 1 This is a functional block diagram of a vehicle 100 provided in an embodiment of this application. In one embodiment, the vehicle 100 is configured in a fully or partially autonomous driving mode. For example, the vehicle 100 can control itself while in autonomous driving mode, and can determine the current state of the vehicle and its surrounding environment through human operation, determine the possible behaviors of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the probability of that other vehicle performing the possible behavior, and control the vehicle 100 based on the determined information. When the vehicle 100 is in autonomous driving mode, the vehicle 100 can be set to operate without human interaction.

[0040] Vehicle 100 may include various subsystems, such as a mobility system 102, a sensing system 104, a control system 106, one or more peripheral devices 108, a power supply 110, a computer system 112, and a user interface 116. Optionally, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 100 may be interconnected via wired or wireless means.

[0041] The propulsion system 102 may include components that provide powered motion to the vehicle 100. In one embodiment, the propulsion system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels (or tires) 121. The engine 118 may be an internal combustion engine, an electric motor, an air-compressed engine, or other types of engine combinations, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. The engine 118 converts the energy source 119 into mechanical energy.

[0042] Examples of energy sources 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 119 may also provide energy to other systems of vehicle 100.

[0043] The transmission 120 can transmit mechanical power from the engine 118 to the wheels 121. The transmission 120 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 120 may also include other components, such as a clutch. The drive shaft may include one or more axles that can be coupled to one or more wheels 121.

[0044] The sensing system 104 may include several sensors for sensing information about the environment surrounding the vehicle 100. For example, the sensing system 104 may include a global positioning system 122 (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and a camera 130. The sensing system 104 may also include sensors for the internal systems of the monitored vehicle 100 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). These detected objects include the detection and identification of road structures and other targets located on the road, a critical function for the safe operation of the autonomous vehicle 100.

[0045] The positioning system 122 can be used to estimate the geographical location of the vehicle 100. The IMU 124 is used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 124 can be a combination of an accelerometer and a gyroscope.

[0046] Radar 126 can use radio signals to sense objects in the surrounding environment of vehicle 100. In some embodiments, in addition to sensing objects, radar 126 can also be used to sense the speed and / or direction of travel of objects.

[0047] The laser rangefinder 128 can use lasers to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0048] Camera 130 can be used to capture multiple images of the surrounding environment of vehicle 100. Camera 130 can be a still camera or a video camera.

[0049] The control system 106 can be configured to control the operation of the vehicle 100 and its components. The control system 106 may include various components, such as a steering system 132, a throttle 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a route control system 142, and an obstacle avoidance system 144.

[0050] The steering system 132 can be used to adjust the forward direction of the vehicle 100. For example, in one embodiment, the steering system 132 can be a steering wheel system.

[0051] Throttle 134 can be used to control the operating speed of engine 118 and thus the speed of vehicle 100.

[0052] Braking unit 136 can be used to control the deceleration of vehicle 100. Braking unit 136 can use friction to slow down wheel 121. In other embodiments, braking unit 136 can convert the kinetic energy of wheel 121 into electrical current. Braking unit 136 can also take other forms to slow down the rotational speed of wheel 121 to control the speed of vehicle 100.

[0053] The computer vision system 140 can be used to process and analyze information such as images captured by the camera 130 and road information reflection points provided by the sensing system, in order to identify objects and / or features in the environment surrounding the vehicle 100. The objects and / or features may include traffic signals, road edges, and obstacles. The computer vision system 140 may use object recognition algorithms, structure-from-motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 can be used to map the environment, track objects, estimate object speeds, etc.

[0054] The route control system 142 is used to determine the driving route of the vehicle 100. In some embodiments, the route control system 142 may combine data from sensor fusion algorithm 138, global positioning system 122 and one or more predetermined maps to determine the driving route of the vehicle 100.

[0055] The obstacle avoidance system 144 is used to identify, assess, and avoid or otherwise traverse potential obstacles in the environment of the vehicle 100.

[0056] Of course, in one embodiment, the control system 106 may additionally or alternatively include, in addition to Figure 1 Components other than those shown and described, or perhaps a portion of them, may be reduced. Figure 1 The components shown.

[0057] Vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users via peripheral devices 108. Peripheral devices 108 may include a wireless communication system 146, an on-board computer 148, a microphone 150, and / or a speaker 152.

[0058] In some embodiments, peripheral device 108 provides a means for a user of vehicle 100 to interact with user interface 116. For example, on-board computer 148 may provide information to a user of vehicle 100. User interface 116 may also operate on-board computer 148 to receive user input. On-board computer 148 may be operated via a touchscreen. In other cases, peripheral device 108 may provide a means for vehicle 100 to communicate with other devices located within the vehicle. For example, microphone 150 may receive audio (e.g., voice commands or other audio input) from a user of vehicle 100. Similarly, speaker 152 may output audio to a user of vehicle 100.

[0059] The wireless communication system 146 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless communication system 146 can communicate using WiFi and a wireless local area network (WLAN). In some embodiments, the wireless communication system 146 can communicate directly with devices using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicle communication systems, may also be used. For example, the wireless communication system 146 may include one or more dedicated short-range communications (DSRC) devices, which may include public and / or private data communications between vehicles and / or roadside stations.

[0060] Power source 110 can provide power to various components of vehicle 100. In one embodiment, power source 110 can be a rechargeable lithium-ion or lead-acid battery. One or more such battery packs can be configured to provide power to various components of vehicle 100. In some embodiments, power source 110 and energy source 119 can be implemented together, as is the case in some fully electric vehicles.

[0061] Some or all of the functions of vehicle 100 are controlled by computer system 112. Computer system 112 may include at least one processor 113, which executes instructions 115 stored in a non-transitory computer-readable medium such as memory 114. Computer system 112 may also be multiple computing devices that control individual components or subsystems of vehicle 100 in a distributed manner.

[0062] Processor 113 can be any conventional processor, such as a commercially available CPU. Alternatively, the processor can be a special-purpose device such as an ASIC or other hardware-based processor. Although Figure 1 The processor, memory, and other elements in the computer system 112 within the same block are functionally illustrated; however, those skilled in the art will understand that the processor, computer, or memory may or may not be stored in the same physical enclosure. For example, memory may be a hard disk drive or other storage media located in an enclosure different from that of computer system 112. Therefore, references to processors or computers will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor that performs calculations only related to the component's specific function.

[0063] In the various aspects described herein, the processor may be located remotely from the vehicle and communicate wirelessly with the vehicle. In other aspects, some of the processes described herein are executed on a processor located within the vehicle, while others are executed by a remote processor, including taking the necessary steps to perform a single operation.

[0064] In some embodiments, memory 114 may contain instructions 115 (e.g., program logic) that can be executed by processor 113 to perform various functions of vehicle 100, including those described above. Memory 114 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the mobility system 102, sensing system 104, control system 106, and peripheral devices 108.

[0065] In addition to instruction 115, memory 114 may also store data such as road maps, route information, vehicle position, direction, speed, and other vehicle data, as well as other information. This information can be used by vehicle 100 and computer system 112 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0066] For example, after radar 126 scans the vehicle's surroundings, memory 114 can store point cloud data formed by the reflection points. Vehicle 100 or computer system 112 can then identify lane line reflection points from the point cloud to determine the lane line's location. Subsequently, vehicle 100 or computer system 112 can locate or control vehicle 100 based on the lane lines.

[0067] User interface 116 is used to provide information to or receive information from users of vehicle 100. Optionally, user interface 116 may include one or more input / output devices within a set of peripheral devices 108, such as wireless communication system 146, on-board computer 148, microphone 150, and speaker 152.

[0068] Computer system 112 can control the functions of vehicle 100 based on input received from various subsystems (e.g., driving system 102, sensing system 104, and control system 106) and from user interface 116. For example, computer system 112 can utilize input from control system 106 to control steering system 132 to avoid obstacles detected by sensing system 104 and obstacle avoidance system 144. In some embodiments, computer system 112 is operable to provide control over many aspects of vehicle 100 and its subsystems.

[0069] Alternatively, one or more of these components may be installed separately from or associated with vehicle 100. For example, memory 114 may exist partially or completely separately from vehicle 100. The components may be communicatively coupled together in a wired and / or wireless manner.

[0070] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 1 This should not be construed as a limitation on the embodiments of this application.

[0071] Autonomous vehicles traveling on roads, such as vehicle 100 above, can identify objects in their surrounding environment to determine adjustments to their current speed. These objects can be other vehicles, traffic control equipment, or other types of objects. In some examples, each identified object can be considered independently, and based on the object's individual characteristics, such as its current speed, acceleration, and distance from the vehicle, the speed adjustment to be made by the autonomous vehicle can be determined.

[0072] Optionally, the autonomous vehicle 100 or the computing device associated with the autonomous vehicle 100 (such as...) Figure 1The computer system 112, computer vision system 140, and memory 114 can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all identified objects can also be considered together to predict the behavior of a single identified object. The vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine the degree of stability to which the vehicle will need to adjust (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered in determining the speed of the vehicle 100, such as the lateral position of the vehicle 100 in the road (along a direction perpendicular to the direction of road extension), the curvature of the road, the proximity of static and dynamic objects, etc.

[0073] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., vehicles in adjacent lanes on the road).

[0074] The aforementioned vehicle 100 can be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, and handcart, etc., and this application embodiment does not impose any special limitations.

[0075] The lane line detection method provided in this application can be applied to the autonomous driving system of a vehicle. The following is a combination of... Figure 2 and Figure 3 A detailed explanation of the autonomous driving system is provided. It should be noted that... Figure 2 and Figure 3 The LiDAR 210 in the text can belong to Figure 1 Part of the sensing system 104, for example, could be Figure 1 A specific form of radar 126 in the system. Figure 2 The environmental perception module 220, positioning module 230, and planning and control module 240 can be composed of Figure 1 The control system 106 in the middle can be implemented, or it can be implemented by Figure 1 The computer system 112 in the middle can be implemented, or it can be implemented by both. Figure 2 The acceleration / deceleration / steering device 250 can be part of the travel system 102.

[0076] See Figure 2The lidar 210 (or lidar sensor) can be used to scan the environment around the vehicle. For example, the lidar can scan the road surface on which the vehicle is traveling. The lidar 210 can then transmit the received scan information to the environmental perception module 220 and the positioning module 230. This scan information may include, for example, information about reflection points formed by obstacles around the vehicle (such as the road surface or other vehicles) reflecting the laser emitted by the lidar.

[0077] The positioning module 230 can process the scanned information to sense the vehicle's position and orientation on the road.

[0078] See Figure 3 The environmental perception module 220 may include a target object perception module 221, a drivable area perception module 222, and a road structure information perception module 223.

[0079] The target object perception module 221 can detect information about other vehicles on the road based on the scanning information input from the lidar 210. For example, it can detect information that reflects the distance between vehicles.

[0080] The driving area perception module 222 can detect information such as road conditions and vehicle driving areas based on the scanning information input by the lidar 210.

[0081] The road structure information perception module 223 can detect the road structure based on the scanning information input from the lidar 210. The road structure information perception module 223 may include a lane line perception module 224. This lane line perception module 224 can process the scanning information to extract information about lane lines, thereby achieving lane line perception.

[0082] The planning and control module 240 can receive the processing results from the environmental perception module 220 and the positioning module 240, and analyze the processing results to control the acceleration, deceleration, and steering devices 250. For example, the planning and control module 240 can make accurate path planning based on the lane line information perceived by the lane line perception module 224, and control the vehicle's driving state through the acceleration and steering devices 250.

[0083] As described above, accurate lane line detection is crucial for autonomous driving. To detect lane lines, the technology first uses LiDAR to scan the vehicle's surroundings and extracts candidate lane line reflection points based on the intensity of the laser reflections. Since these candidate reflection points contain a significant amount of noise, the technology filters out this noise to obtain the final lane line reflection points. The lane line's position can then be determined based on these reflection points.

[0084] However, the related technologies do not make full use of road edge information when extracting lane line reflection points from candidate lane line reflection points, and the related technologies mainly extract lane line reflection points in the vehicle coordinate system. These factors all lead to the extraction of lane line reflection points that are not accurate and reliable enough.

[0085] The following is combined Figure 4 The lane line detection method provided in the embodiments of this application will be described in detail. Figure 4 The provided lane line detection method can be applied to autonomous driving systems in vehicles, for example, it can be applied to... Figure 2 In the environmental perception module 220 of the autonomous driving system shown. For example... Figure 4 As shown, the method includes steps S410-S440, and each step is described in detail below.

[0086] In step S410, the LiDAR is used to scan the vehicle's surrounding environment to obtain lane line candidate reflection points and road edge information. For example, firstly, the LiDAR is used to scan the vehicle's surrounding environment and obtain the emission points. Then, lane line candidate reflection points and road edge information can be obtained based on the intensity information of the reflection points and the characteristics of the reflection points and adjacent reflection points. For a detailed implementation of step S410, see [link to implementation details]. Figure 5 .

[0087] In step S420, a road edge coordinate system is established based on the road edge information. The road edge information can be information about the road edge lines, such as their location, length, and differences between two road edge lines. The road edge coordinate system can be a two-dimensional Cartesian coordinate system established with the road edge lines as a reference line, or it can be a Frenet coordinate system. For example, the road edge coordinate system can be a coordinate system established with the road edge lines as one axis of a two-dimensional Cartesian coordinate system.

[0088] In step S430, lane line reflection points are extracted from the candidate lane line reflection points based on their coordinates in the road edge coordinate system. Before executing step S430, the candidate lane line reflection points can be projected from the current coordinate system (such as the vehicle coordinate system) to the road edge coordinate system, and then processed in the road edge coordinate system.

[0089] There are several ways to implement step S430. For example, one can first determine the features of the candidate lane line reflection points based on their coordinates in the road edge coordinate system; then, based on these features, extract the lane line reflection points from the candidate lane line reflection points. The features of the candidate lane line reflection points include one or more of the following: the orientation of the candidate lane line reflection points, the distance between the candidate lane line reflection points and the road edge line, the spacing between the candidate lane line reflection points, and whether the candidate lane line reflection points can form a straight line, etc. Alternatively, the candidate lane line reflection points can be clustered in the road edge coordinate system, and then the lane line reflection points can be extracted from the candidate lane line reflection points based on the clustering results. (The following text...) Figures 8 to 10 Several possible implementations of step S430 are given.

[0090] In step S440, the lane line is obtained based on the lane line reflection points. For example, the lane line reflection points can be transformed to the vehicle coordinate system, and then the lane line is fitted in the vehicle coordinate system. After obtaining the lane line, it can be output directly, or it can be tracked before being output. The lane line information obtained in step S440 can be output to, for example,... Figure 2 The planning and control module 240 shown is used for vehicle path planning and / or steering control.

[0091] Since there is a certain correlation between road edges and lane lines, using road edge information as prior information can effectively filter out noise in candidate lane line reflection points, improving the accuracy and stability of lane line detection. Furthermore, regardless of the actual shape of the lane lines, the arrangement of lane line reflection points in the road edge coordinate system is relatively regular; therefore, extracting lane line reflection points in the road edge coordinate system is also simpler.

[0092] The following is combined Figure 5 One possible implementation of step S410 is given. Figure 5 This includes steps S510 to S530, which are described in detail below.

[0093] In step S510, point cloud acquisition and preprocessing are performed.

[0094] There are various methods for acquiring and preprocessing point clouds. For example, in some embodiments, a lidar can be used to scan the surrounding environment with one or more layers to acquire point clouds formed by reflection points. Then, a ground model is built based on the point cloud, and some obvious noise points are filtered out based on the ground model.

[0095] As an example, reflection points in the point cloud can be projected onto a 2D grid, and the minimum and maximum heights of the reflection points projected onto each grid relative to the ground can be calculated. If the difference between the maximum and minimum heights is less than a certain threshold (e.g., 0.1m), the minimum height of that grid relative to the ground can be considered the ground height of that grid. Once the ground heights of all grids are determined, a ground model is established. Then, based on the ground model, the height of each reflection point relative to the ground can be calculated, and reflection points with heights exceeding a certain threshold (e.g., 2m) can be filtered out.

[0096] In step S520, road edge information is extracted. For example, in some embodiments, candidate reflection points of the road edge can be extracted based on the features of each reflection point and its adjacent points in the point cloud; then, noise points in the candidate reflection points of the road edge can be removed to obtain road edge line reflection points; then, the road edge line can be detected based on the road edge line reflection points to obtain road edge information.

[0097] As an example, the following steps can be used to extract road edge information.

[0098] Step a: Extract candidate reflection points for road edges from the point cloud. For example, candidate reflection points for road edges can be extracted based on the features of the reflection points and their adjacent reflection points. The features of the reflection points and their adjacent reflection points may include one or more of the following features: height variation, angle variation, distance difference, etc. between the reflection points.

[0099] Step b: Cluster the candidate reflection points at the road edge and filter out clusters with fewer than a certain threshold of reflection points.

[0100] Step c: Perform least-squares fitting on each cluster to form a fitted line. Then, calculate the distance between each reflection point in the cluster and the fitted line, and remove one or more reflection points with the largest or largest distance from the cluster. This improves the accuracy of the fitted line, thereby increasing the reliability of the subsequently extracted road edge information.

[0101] Step d: Repeat step c until the distance between all reflection points of the candidate reflection points at the road edge and the corresponding fitted line is less than a certain threshold.

[0102] Step e: Output the fitted line as the road edge line.

[0103] In step S530, candidate reflection points for lane lines are extracted. For example, in some embodiments, candidate reflection points for lane lines can be extracted based on the features of each reflection point in the point cloud and adjacent reflection points.

[0104] As an example, the following steps can be used to extract candidate reflection points for lane lines.

[0105] Step a: For a reflection point, determine the height of the reflection point relative to the ground based on the ground model established in step S510. If the height of the reflection point meets a certain threshold (e.g., 0.1m), proceed to step b; otherwise, select the next reflection point.

[0106] Step b: Take adjacent reflection points within a certain range of the reflection point, and calculate the maximum reflection intensity among the reflection points within that range. With minimum reflection intensity For example, we can take the reflection point and five reflection points on each side, for a total of 11 reflection points, and calculate the maximum reflection intensity among the 11 reflection points. With minimum reflection intensity

[0107] Step c: Determine the reflection intensity at the reflection point. Does it meet the requirements? Where β is a preset threshold, for example, it can be 0.9. If the above condition is met, the reflection point belongs to the lane line candidate reflection point; if not, the next reflection point is taken, and step ac is repeated until all reflection points in the point cloud have been traversed.

[0108] See again Figure 4 There are several ways to establish the road edge coordinate system mentioned in step S420. The following will combine... Figure 6 and Figure 7 Please provide a detailed description.

[0109] Figure 6 This is a vehicle coordinate system established based on the vehicle's position and direction of travel, where the positive direction of the x-axis is the vehicle's forward direction, and the y-axis is perpendicular to the vehicle's forward direction. Figure 6 In the vehicle coordinate system shown, the two road edge lines are road edge lines a and b, and both road edge lines a and b are curves.

[0110] In some embodiments, it can be used as Figure 6 Using the road edge line b as a reference line, establish a road edge coordinate system. The established road edge coordinate system is as follows: Figure 7 As shown. In Figure 7 In the diagram, the x' axis represents the road edge line b. Figure 7 It can be seen that the candidate reflection points of the lane lines, which were originally arranged according to the curve, are now... Figure 7 The lane lines are arranged in a straight line. Therefore, extracting lane line reflection points in this road edge coordinate system is simpler to implement.

[0111] Figure 7A road edge coordinate system is established using one of the two road edge lines as a reference line. When the two road edge lines differ in length, the longer of the two road edge lines can be used as the reference line. Actual road edge lines are usually longer, therefore the longer road edge line is more likely to be the actual road edge line, and the road edge coordinate system established using this longer edge line as the reference line is more reliable.

[0112] Furthermore, in some embodiments, a road edge coordinate system can be established using the centerline of the two road edge lines as a reference line. Alternatively, in some embodiments, if the two road edge lines differ significantly, two separate coordinate systems can be established using the two road edge lines as reference lines, and the lane line reflection points selected from the two coordinate systems can be merged. When the two road edge lines differ significantly, using two coordinate systems to separately filter candidate lane line reflection points reduces the likelihood of missing lane line reflection points, resulting in more accurate lane line detection results.

[0113] See again Figure 4 There are several ways to extract lane line reflection points as mentioned in step S430. The following will combine... Figures 8-10 Several possible implementations of step S430 are given.

[0114] Figure 8 for Figure 4 A flowchart illustrating one possible implementation of step S430 in the above steps. Figure 8 The steps include steps S810 and S820.

[0115] In step S810, based on the coordinates of the candidate reflection points of the lane line in the road edge coordinate system, feature detection is performed by Hough transform to obtain a straight line.

[0116] In the road edge coordinate system, lane lines tend to have a relatively regular shape, such as a straight line. Therefore, the Hough transform can be used to find the straight-line characteristics of candidate reflection points for lane lines.

[0117] In step S820, reflection points whose distance from the straight line is less than a threshold are extracted from the candidate reflection points of the lane line and used as lane line reflection points. The distance between the candidate reflection point of the lane line and the straight line can be the distance between the horizontal coordinate and / or vertical coordinate of the candidate reflection point of the lane line and the straight line.

[0118] The following is combined Figure 6 and Figure 7 , give Figure 8 One specific implementation of the embodiment.

[0119] like Figure 6As shown, road edge lines a and b are curves in the actual environment. These curves can be represented by quadratic or higher-order equations. For example, the expression for road edge line a is y = 10 + 0.1x + 0.01x. 2 The expression for the other road edge line b is y = -10 + 0.1x + 0.01x. 2 .

[0120] Assuming the coordinates of the candidate reflection point of the lane line are (x, y) in the vehicle coordinate system (the unit of coordinate values ​​can be meters), then in the following... Figure 7 In the road edge coordinate system shown, the coordinates of the candidate reflection point of the lane line are transformed into (x', y'), and the numerical relationship between (x', y') and (x, y) satisfies:

[0121] x'=x

[0122] y' = y - (-10 + 0.1x + 0.01x) 2 )

[0123] Assumption Figure 6 The coordinates of the candidate reflection points of the lane line in the coordinate system are as follows:

[0124] (x = 0, y = 0)

[0125] (x = 10, y = 2)

[0126] (x = 20, y = 6)

[0127] (x = 30, y = 12)

[0128] (x = 40, y = 20)

[0129] (x = 40, y = 23)

[0130] (x = 50, y = 30)

[0131] Project these lane line candidate reflection points onto Figure 7 After establishing the road edge coordinate system, the coordinates are as follows:

[0132] (x' = 0, y' = 10)

[0133] (x' = 10, y' = 10)

[0134] (x' = 20, y' = 10)

[0135] (x' = 30, y' = 10)

[0136] (x' = 40, y' = 10)

[0137] (x' = 40, y' = 13)

[0138] (x' = 50, y' = 10)

[0139] like Figure 7 As shown, in the road edge coordinate system, a straight line c is first found using the Hough transform. The endpoints of the straight line c are (0, 10) and (50, 10).

[0140] Then, the distance from each candidate lane line reflection point to the straight line c is calculated. Points with a distance less than a certain preset threshold (e.g., 0.1m) are marked as lane line reflection points, and reflection points farther from the straight line are removed. Figure 7 The points in the middle (x' = 40, y' = 13). Finally, the lane line reflection points extracted from the lane line candidate reflection points are:

[0141] (x' = 0, y' = 10)

[0142] (x' = 10, y' = 10)

[0143] (x' = 20, y' = 10)

[0144] (x' = 30, y' = 10)

[0145] (x' = 40, y' = 10)

[0146] (x' = 50, y' = 10)

[0147] The above text combined Figure 8 This paper details how to extract lane line reflection points using the Hough transform in a road edge coordinate system. The following section combines... Figure 9 and Figure 10 This describes in detail how to extract lane line reflection points using clustering in the road edge coordinate system.

[0148] Figure 9 for Figure 4 A flowchart illustrating another possible implementation of step S430 in the above steps. Figure 9 The steps S910-S940 are described in detail below.

[0149] In step S910, a candidate reflection point of a lane line is selected as seed point A, and a new cluster M1 is created. A belongs to M1.

[0150] In step S920, the remaining lane line candidate reflection points are traversed. For example, starting with lane line candidate reflection point B, the relative lateral distance and / or relative longitudinal distance between this candidate reflection point and lane line candidate reflection point A in cluster M1 are determined. If the preset lateral distance threshold and / or longitudinal distance threshold are met, lane line candidate reflection point B is assigned to cluster M1. The process continues to determine the next lane line candidate reflection point, such as lane line candidate reflection point C. This process is repeated sequentially.

[0151] In step S930, select a lane line candidate reflection point from the lane line candidate reflection points that have not been assigned to any cluster, create a new cluster M2, and repeat step S920.

[0152] In step S940, step S930 is repeated until all lane line candidate reflection points are traversed.

[0153] The reflection points in each cluster are the lane line reflection points.

[0154] Optionally, in some embodiments, quadratic or cubic curve fitting can be performed on candidate reflection points in the cluster to filter out candidate reflection points in the cluster that are laterally distant from the fitted curve.

[0155] Optionally, in some embodiments, clusters with fewer candidate reflection points can be selected, or clusters with shorter distribution distances in the x-direction of the road edge coordinate system can be filtered out.

[0156] Optionally, in some embodiments, clusters can be filtered out or merged based on their lateral spacing.

[0157] Figure 10 for Figure 4 A flowchart illustrating another possible implementation of step S430 in the above steps. Figure 10 The steps S1010-S1040 are described in detail below.

[0158] In step S1010, initial parameters are set, and the Hough transform is used to extract a set of reflection points that satisfy an approximate linear relationship. The initial parameters may include the number of points in the set of reflection points, or the distance between adjacent points in the set of reflection points.

[0159] In step S1020, the set of reflection points among the candidate reflection points for lane lines whose orientation differs significantly from that of the road edge line is filtered out. Optionally, in some embodiments, the set of reflection points that is significantly spaced from other sets of reflection points can be deleted based on the lateral spacing of the reflection point sets, or some sets of reflection points that are closely spaced can be merged.

[0160] In step S1030, the remaining multiple sets of reflection points that satisfy the approximate straight-line relationship are respectively used as initial clusters, and the candidate reflection points of the lane line that are not in any cluster are traversed.

[0161] For example, for a candidate reflection point C of a lane line, determine the relative lateral distance and / or relative longitudinal distance between candidate reflection point C and all reflection points in the initial cluster (or, the relative normal distance and relative tangential distance can be used instead of the relative lateral distance and / or relative longitudinal distance). If the relative lateral distance and / or relative longitudinal distance between candidate reflection point C and reflection points in cluster M1 of the initial cluster meets the set lateral distance threshold and / or longitudinal distance threshold, then further calculate the number of points in cluster M1 that meet the set lateral distance threshold and / or longitudinal distance threshold. If the number of reflection points in cluster M1 meets the set threshold, then point C can be assigned to cluster M1. And so on.

[0162] Optionally, in some embodiments, in addition to traversing the candidate reflection points of the lane lines, other reflection points in the initially obtained point cloud can also be traversed, such as reflection points that are close to the initial cluster.

[0163] After the above process, the reflection points in each cluster are the lane line reflection points.

[0164] See again Figure 4 In some embodiments, after obtaining road edge information, the reliability of the road edge information can be determined first. If the road edge information is reliable, the scheme provided in this application embodiment can be used to extract lane line reflection points in the road edge coordinate system; otherwise, lane line reflection points can be extracted in the vehicle coordinate system. There are various ways to measure whether road edge information is reliable. For example, the reliability of road edge information can be determined by judging whether two road edge lines are long enough or whether the difference between two road edge lines is small enough.

[0165] It should be noted that the scanning using LiDAR in step S410 can be a single-frame scan or a multi-frame scan. Using the multi-frame scan results of LiDAR to extract road edge information or lane line information can improve the accuracy of the extraction results.

[0166] The device embodiments of this application are described below. Since the device embodiments correspond to the method embodiments, the parts not described in detail can be referred to the preceding method embodiments.

[0167] Figure 11 This is a schematic diagram of the lane line detection device provided in an embodiment of this application. Figure 11As shown, the detection device 1100 includes a scanning module 1110, an establishment module 1120, an extraction module 1130, and a determination module 1140. The scanning module 1110 uses a lidar to scan the vehicle's surrounding environment, obtaining lane line candidate reflection points and road edge information. The establishment module 1120 establishes a road edge coordinate system based on the road edge information. The extraction module 1130 extracts lane line reflection points from the lane line candidate reflection points based on their coordinates in the road edge coordinate system. The determination module 1140 determines the lane lines based on the lane line reflection points.

[0168] Optionally, in some embodiments, the extraction module 1120 can be used to determine the features of the lane line candidate reflection points based on the coordinates of the lane line candidate reflection points in the road edge coordinate system; and to extract lane line reflection points from the lane line candidate reflection points based on the features of the lane line candidate reflection points.

[0169] Optionally, in some embodiments, the extraction module 1120 can be used to perform feature detection through Hough transform based on the coordinates of the lane line candidate reflection points in the road edge coordinate system to obtain a straight line; and extract reflection points from the lane line candidate reflection points whose distance from the straight line is less than a first threshold as lane line reflection points.

[0170] Optionally, in some embodiments, the extraction module 1120 can be used to cluster the candidate lane line reflection points based on the coordinates of the candidate lane line reflection points in the road edge coordinate system; and determine the lane line reflection points based on the clustering results.

[0171] Optionally, in some embodiments, the detection device 1100 may further include: a filtering module, used to filter out reflection points from the candidate reflection points of lane lines that are at a distance greater than a second threshold from the road edge, based on road edge information, before extracting lane line reflection points from the candidate reflection points of lane lines based on the coordinates of the candidate reflection points of lane lines in the road edge coordinate system.

[0172] Optionally, in some embodiments, the road edge information includes information about two road edge lines, and the establishment module 1120 can be used to establish a road edge coordinate system using the longer of the two road edge lines or the center line of the two edge lines as a reference line.

[0173] Optionally, in some embodiments, the road edge information includes information about two road edge lines that differ from each other, and the road edge coordinate system includes two coordinate systems established with the two road edge lines as reference lines respectively.

[0174] The detection device provided in the embodiments of this application includes a memory and at least one processor. For example... Figure 12 As shown, the testing device 1200 may include, for example: Figure 1 The memory 114 shown is connected to at least one processor 113. The memory 114 stores instructions 115 that are executed by at least one processor 113 to cause the at least one processor 113 to perform the lane line detection method described above. For example, the processor and the memory can be connected to a bus, or they can be connected to other external devices via the bus.

[0175] For example, the detection device provided in the embodiments of this application may include at least one lidar and a processor. The lidar is used to acquire point cloud data of the vehicle's surrounding environment, and the processor is used to run software programs, including a lidar driver, a lidar lane line perception module, and a lane line receiving module. The lidar driver is used to convert the data sent by the lidar to the processor into three-dimensional point cloud data. The lidar lane line perception module includes an extraction module, which is used to extract lane lines based on road edge information (road edge coordinate system). The lane line receiving module may be a fusion, planning, or control module, used to receive the lane line results. If multi-frame accumulation or tracking is required, a vehicle motion information acquisition module is also needed to provide vehicle motion information, which may come from a positioning module or utilize vehicle wheel speed sensors, steering wheel angle, etc.

[0176] For example, in some embodiments, the detection device may further include interfaces for connecting the various components, which may include high-speed interfaces and low-speed interfaces. The various components may be interconnected using different buses and may be mounted on a common motherboard or otherwise installed as needed. The processor 113 may process instructions 115 executed within the detection device 1300, including instructions stored in or on memory 114 for displaying graphical information on an external input / output device (such as a display device coupled to the interface).

[0177] For example, in some embodiments, the detection device can be a digital computer of various forms that can be fixed in the vehicle, such as a laptop computer, desktop computer, workbench, personal digital assistant, server, blade server, mainframe computer, and other suitable computer. The detection device can also be a mobile device of various forms that can be wirelessly connected to the vehicle's signal, such as a personal digital processor, cellular phone, smartphone, wearable device, and other similar computing device.

[0178] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0179] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, devices, storage media, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0180] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0181] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0182] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of detecting a lane line, characterized by, The method comprises: scanning the surrounding environment of the vehicle by using a laser radar to obtain lane line candidate reflection points and road edge information; establishing a road edge coordinate system based on the road edge information; performing feature detection on the lane line candidate reflection points in the road edge coordinate system by using a Hough transform to obtain a straight line based on the coordinates of the lane line candidate reflection points in the road edge coordinate system; extracting, from the lane line candidate reflection points, reflection points with a distance less than a first threshold from the straight line as lane line reflection points; and obtaining a lane line based on the lane line reflection points. Before the step of performing feature detection on the lane line candidate reflection points in the road edge coordinate system by using a Hough transform to obtain a straight line based on the coordinates of the lane line candidate reflection points in the road edge coordinate system, the method further comprises:

2. The detection method according to claim 1, characterized in that, filtering, from the lane line candidate reflection points, reflection points with a distance greater than a second threshold from a road edge based on the road edge information. The road edge information comprises information of two road edge lines, 3. The detection method according to claim 1 or 2, characterized in that, The step of establishing a road edge coordinate system based on the road edge information comprises: establishing the road edge coordinate system by taking a longer one of the two road edge lines or a center line of the two road edge lines as a reference line. The road edge information comprises information of two road edge lines, and the two road edge lines are different, and the road edge coordinate system comprises two coordinate systems respectively established by taking the two road edge lines as reference lines.

4. The detection method according to claim 1 or 2, characterized in that, The method comprises:

5. A lane line detection device characterized by comprising: a scanning module configured to scan the surrounding environment of the vehicle by using a laser radar to obtain lane line candidate reflection points and road edge information; an establishing module configured to establish a road edge coordinate system based on the road edge information; an extracting module configured to perform feature detection on the lane line candidate reflection points in the road edge coordinate system by using a Hough transform to obtain a straight line based on the coordinates of the lane line candidate reflection points in the road edge coordinate system; and extract, from the lane line candidate reflection points, reflection points with a distance less than a first threshold from the straight line as lane line reflection points; and a determining module configured to obtain a lane line based on the lane line reflection points. The detection device further comprises: a filtering module configured to filter, from the lane line candidate reflection points, reflection points with a distance greater than a second threshold from a road edge based on the road edge information before the step of performing feature detection on the lane line candidate reflection points in the road edge coordinate system by using a Hough transform to obtain a straight line based on the coordinates of the lane line candidate reflection points in the road edge coordinate system.

6. The detection device of claim 5, wherein, The road edge information comprises information of two road edge lines, and the establishing module is configured to establish the road edge coordinate system by taking a longer one of the two road edge lines or a center line of the two road edge lines as a reference line. The road edge information comprises information of two road edge lines, and the two road edge lines are different, and the road edge coordinate system comprises two coordinate systems respectively established by taking the two road edge lines as reference lines.

7. The detection device according to claim 5 or 6, characterized in that The method comprises:

8. The detection device according to claim 5 or 6, characterized in that at least one processor; 9. A lane line detection device characterized by comprising: and a memory connected with the at least one processor. ​ ​ The memory stores instructions executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the lane line detection method in any one of claims 1-4.

10. A chip, characterized by The chip comprises a processor and a memory, and the memory stores instructions, and the processor is configured to execute the instructions stored in the memory, and when the instructions are executed, the processor is configured to perform the lane line detection method in any one of claims 1-4.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores instructions, and when the instructions are executed on the computer, the computer is enabled to perform the lane line detection method in any one of claims 1-4.

12. A computer program product comprising instructions, characterized in that, When the instructions are executed on the computer, the computer is enabled to perform the lane line detection method in any one of claims 1-4.

Images