Road surface marking recognition method and device

Abstract

A method and device for road surface marking recognition, the method comprising: obtaining three-dimensional point cloud data detected by a lidar, the three-dimensional point cloud data including reflection data of a road surface marking area (401); compressing the three groups of point cloud data into a two-dimensional point cloud feature map (402); processing the two-dimensional point cloud feature map to obtain a road surface marking recognition result (403). This method enables road surface marking recognition to be no longer limited by the imaging environment and expands the usage scenarios of road surface marking recognition.

Description

Technical Field

[0001] This application relates to the technical field of automobiles, and particularly to a method and device for road sign recognition. Background Art

[0002] With the continuous development of automotive technology, the demand for road sign recognition is also increasing.

[0003] Generally, the recognition result of road signs is obtained by processing the images collected by an image acquisition device, which can be set on a vehicle. Since the method of road sign recognition based on images is greatly affected by the imaging quality, when the imaging quality is good, due to the large amount of useful information in the image, a relatively accurate recognition result can be obtained. When the imaging quality is poor, due to the small amount of useful information in the image, the recognition accuracy is low. When the imaging quality is very low, it is no longer possible to be used for road sign recognition.

[0004] Therefore, the above method of road sign recognition based on images has limited application scenarios and is difficult to meet the needs of users. Summary of the Invention

[0005] Embodiments of this application provide a method and device for road sign recognition to solve the problem of large limitations in the existing method of road sign recognition based on images.

[0006] In a first aspect, embodiments of this application provide a method for road sign recognition, including:

[0007] Obtaining three-dimensional point cloud data detected by a lidar, where the three-dimensional point cloud data includes reflection data of a road sign area;

[0008] Compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map;

[0009] Processing the two-dimensional point cloud feature map to obtain a road sign recognition result.

[0010] In a second aspect, embodiments of this application provide a road sign recognition device, including: a processor and a memory; the memory is used to store program code; the processor, when calling the program code and the program code is executed, is used to perform the following operations:

[0011] Obtaining three-dimensional point cloud data detected by a lidar, where the three-dimensional point cloud data includes reflection data of a road sign area;

[0012] Compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map;

[0013] Processing the two-dimensional point cloud feature map to obtain a road sign recognition result.

[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, where the computer program includes at least one piece of code that can be executed by a computer to control the computer to execute the method according to any one of the first aspects above.

[0015] In a fourth aspect, an embodiment of the present application provides a computer program that, when executed by a computer, is used to implement the method according to any one of the first aspects above.

[0016] An embodiment of the present application provides a road surface marking recognition method and device. By obtaining three-dimensional point cloud data detected by a lidar, the three-dimensional point cloud data includes data of a road surface marking area, compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map, and processing the two-dimensional point cloud feature map to obtain a road surface marking recognition result. Since the lidar has high reliability and is hardly affected by environmental factors, even in a harsh imaging environment, a relatively accurate road surface marking recognition result can be obtained based on the three-dimensional point cloud data obtained by the lidar, enabling the road surface marking recognition to be no longer limited by the imaging environment, expanding the usage scenarios of the road surface marking recognition, and solving the problem that the usage scenarios of the traditional method of roadside marking recognition based on images are relatively limited and difficult to meet user needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1A - Figure 1B It is a schematic diagram of an application scenario of the road surface marking recognition method provided by an embodiment of the present application;

[0019] Figure 1C It is a schematic diagram of the structure of the lidar;

[0020] Figure 2 It is a schematic diagram of the coaxial optical path adopted by the lidar;

[0021] Figure 3 It is a schematic diagram of the scanning pattern of the lidar;

[0022] Figure 4 It is a schematic flowchart of the road surface marking recognition method provided by an embodiment of the present application;

[0023] Figure 5 It is a schematic flowchart of the road surface marking recognition method provided by another embodiment of the present application;

[0024] Figure 6 Schematic flow chart of the road sign recognition method provided by another embodiment of the present application;

[0025] Figure 7 Schematic diagram of the preset neural network model provided by the embodiment of the present application;

[0026] Figure 8 Schematic diagram I of the relationship between the target direction and the road surface distance provided by the embodiment of the present application;

[0027] Figure 9A - Figure 9B Schematic diagram of the compressed shape of the road sign provided by the embodiment of the present application;

[0028] Figure 10 Schematic diagram of the relationship between the target direction and the road surface distance provided by the embodiment of the present application Figure Two ;

[0029] Figure 11 Schematic diagram of the target direction of the curved road surface provided by the embodiment of the present application;

[0030] Figure 12 Schematic diagram of the structure of the road sign recognition device provided by an embodiment of the present application. Detailed implementation manners

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0032] The road sign recognition method provided by the embodiments of the present application can be applied to any scenario that requires road sign recognition. The road sign recognition method can be specifically executed by a road sign recognition device. The road sign recognition device can be a device including a lidar. Correspondingly, the application scenario schematic diagram of the road sign recognition method provided by the embodiments of the present application can be as Figure 1A shown. Specifically, the lidar of the road sign recognition device can detect three-dimensional point cloud data, and the processor of the road sign recognition device can process the three-dimensional point cloud data obtained by the lidar using the road sign recognition method provided by the embodiments of the present application. It should be noted that Figure 1A it is only a schematic diagram and does not limit the structure of the road sign recognition device.

[0033] Alternatively, the road surface marking recognition device may also be a device that does not include a lidar. Correspondingly, the application scenario schematic diagram of the road surface marking recognition method provided in the embodiments of the present application may be as follows Figure 1B shown. Specifically, the communication interface of the road surface marking recognition device may receive the three-dimensional point cloud data obtained by lidar detection sent by other devices or equipment, and the processor of the road surface marking recognition device may process the received three-dimensional point cloud data by using the road surface marking recognition method provided in the embodiments of the present application. It should be noted that Figure 1B is only a schematic diagram, and does not limit the structure of the road surface marking recognition device and the connection manner between the road surface marking recognition device and other devices or equipment. For example, the communication interface in the image processing device may be replaced by a transceiver.

[0034] Among them, the lidar is used to sense external environment information. For example, the distance information, azimuth information, reflection intensity information, speed information, etc. of environmental targets. In one implementation, the lidar may detect the distance from the lidar to the detected object by measuring the time of light propagation between the lidar and the detected object, that is, the time of flight (TOF). Alternatively, the lidar may also detect the distance from the lidar to the detected object by other technologies, such as a ranging method based on phase shift measurement, or a ranging method based on frequency shift measurement, which is not limited here.

[0035] For ease of understanding, the following will be combined with Figure 1C the lidar 100 shown to describe the working process of ranging by way of example. As Figure 1C shown, the lidar 100 may include a transmitting circuit 110, a receiving circuit 120, a sampling circuit 130, and an arithmetic circuit 140.

[0036] The transmitting circuit 110 may transmit a light pulse sequence (such as a laser pulse sequence). The receiving circuit 120 may receive the light pulse sequence reflected by the detected object, perform photoelectric conversion on the light pulse sequence to obtain an electrical signal, and output it to the sampling circuit 130 after processing the electrical signal. The sampling circuit 130 may sample the electrical signal to obtain a sampling result. The arithmetic circuit 140 may determine the distance between the lidar 100 and the detected object based on the sampling result of the sampling circuit 130.

[0037] Optionally, the lidar 100 may further include a control circuit 150, and the control circuit 150 may implement the control of other circuits. For example, it may control the working time of each circuit and / or set parameters for each circuit, etc.

[0038] It should be understood that although Figure 1CThe lidar shown includes a transmitting circuit, a receiving circuit, a sampling circuit, and an arithmetic circuit for emitting a beam of light for detection. However, the embodiments of the present application are not limited thereto. The number of any one of the transmitting circuit, the receiving circuit, the sampling circuit, and the arithmetic circuit may also be at least two for emitting at least two beams of light in the same direction or in different directions respectively. Among them, the at least two beams of light paths may be emitted simultaneously or at different times respectively. In one example, the light-emitting chips in the at least two transmitting circuits are packaged in the same module. For example, each transmitting circuit includes a laser-emitting chip, and the dies of the laser-emitting chips in the at least two transmitting circuits are packaged together and accommodated in the same packaging space.

[0039] In some implementation manners, in addition to Figure 1C the circuits shown, the lidar 100 may further include a scanning module 160 for changing the propagation direction of at least one laser pulse sequence emitted by the transmitting circuit and emitting it.

[0040] Among them, a module including a transmitting circuit 110, a receiving circuit 120, a sampling circuit 130, and an arithmetic circuit 140, or a module including a transmitting circuit 110, a receiving circuit 120, a sampling circuit 130, an arithmetic circuit 140, and a control circuit 150 may be referred to as a ranging module. The ranging module 150 may be independent of other modules, such as the scanning module 160.

[0041] Coaxial light paths may be adopted in the lidar, that is, at least part of the light path is shared between the beam emitted by the lidar and the beam reflected back in the lidar. For example, after at least one laser pulse sequence emitted by the transmitting circuit is emitted after changing the propagation direction by the scanning module, the laser pulse sequence reflected back by the detection object enters the receiving circuit after passing through the scanning module. Or, the lidar may also adopt non-coaxial light paths, that is, the beam emitted by the lidar and the beam reflected back are transmitted along different light paths in the lidar respectively. Figure 2 FIG. shows a schematic diagram of an embodiment in which the lidar of the present application adopts coaxial light paths.

[0042] The lidar 200 includes a ranging module 201. The ranging module 210 includes a transmitter 203 (which may include the above-mentioned transmitting circuit), a collimating element 204, a detector 205 (which may include the above-mentioned receiving circuit, sampling circuit, and arithmetic circuit), and an optical path changing element 206. The ranging module 210 is used to emit a light beam and receive the reflected light, and convert the reflected light into an electrical signal. Among them, the transmitter 203 can be used to emit a light pulse sequence. In one embodiment, the transmitter 203 can emit a laser pulse sequence. Optionally, the laser beam emitted by the transmitter 203 is a narrow-bandwidth beam with a wavelength outside the visible light range. The collimating element 204 is disposed on the outgoing optical path of the transmitter, and is used to collimate the light beam emitted from the transmitter 203, and collimate the light beam emitted from the transmitter 203 into a parallel light beam and emit it to the scanning module. The collimating element is also used to converge at least a part of the reflected light reflected by the detected object. The collimating element 204 can be a collimating lens or other elements capable of collimating light beams.

[0043] In Figure 2 In the illustrated embodiment, the optical path changing element 206 is used to combine the emission optical path and the reception optical path inside the lidar before the collimating element 104, so that the emission optical path and the reception optical path can share the same collimating element, making the optical path more compact. In some other implementation manners, it may also be that the transmitter 103 and the detector 105 respectively use their own collimating elements, and the optical path changing element 206 is disposed on the optical path after the collimating element.

[0044] In Figure 2 In the illustrated embodiment, since the beam aperture of the light beam emitted by the transmitter 103 is small and the beam aperture of the reflected light received by the lidar is large, the optical path changing element can use a small-area mirror to combine the emission optical path and the reception optical path. In some other implementation manners, the optical path changing element can also use a mirror with a through hole, where the through hole is used to transmit the outgoing light of the transmitter 203, and the mirror is used to reflect the reflected light to the detector 205. This can reduce the occlusion of the reflected light by the bracket of the small mirror in the case of using a small mirror.

[0045] In Figure 2 In the illustrated embodiment, the optical path changing element deviates from the optical axis of the collimating element 204. In some other implementation manners, the optical path changing element can also be located on the optical axis of the collimating element 204.

[0046] The lidar 200 further includes a scanning module 202. The scanning module 202 is placed on the outgoing optical path of the ranging module 201. The scanning module 102 is used to change the transmission direction of the collimated light beam 219 emitted by the collimating element 204 and project it to the external environment, and project the reflected light to the collimating element 204. The reflected light is converged onto the detector 105 by the collimating element 104.

[0047] In one embodiment, the scanning module 202 may include at least one optical element for changing the propagation path of a light beam, wherein the optical element can change the propagation path of the light beam by reflecting, refracting, diffracting, etc. the light beam. For example, the scanning module 202 includes a lens, a mirror, a prism, a galvanometer, a grating, a liquid crystal, an optical phased array, or any combination of the above optical elements. In one example, at least part of the optical elements are movable. For example, a driving module is used to drive at least part of the optical elements to move, and the movable optical element can reflect, refract, or diffract the light beam to different directions at different times. In some embodiments, multiple optical elements of the scanning module 202 can rotate or vibrate around a common axis 209, and each rotating or vibrating optical element is used to continuously change the propagation direction of the incident light beam. In one embodiment, multiple optical elements of the scanning module 202 can rotate at different rotational speeds or vibrate at different speeds. In another embodiment, at least part of the optical elements of the scanning module 202 can rotate at substantially the same rotational speed. In some embodiments, multiple optical elements of the scanning module can also rotate around different axes. In some embodiments, multiple optical elements of the scanning module can also rotate in the same direction or in different directions; or vibrate in the same direction or in different directions, which is not limited herein.

[0048] In one embodiment, the scanning module 202 includes a first optical element 214 and a driver 216 connected to the first optical element 214. The driver 216 is used to drive the first optical element 214 to rotate around the rotation axis 209, so that the first optical element 214 changes the direction of the collimated light beam 219. The first optical element 214 projects the collimated light beam 219 to different directions. In one embodiment, the angle between the direction of the collimated light beam 219 after being changed by the first optical element and the rotation axis 109 changes as the first optical element 214 rotates. In one embodiment, the first optical element 214 includes a pair of non-parallel opposite surfaces, and the collimated light beam 219 passes through the pair of surfaces. In one embodiment, the first optical element 214 includes a prism with a thickness varying along at least one radial direction. In one embodiment, the first optical element 114 includes a wedge prism that refracts the collimated light beam 119.

[0049] In one embodiment, the scanning module 202 further includes a second optical element 215. The second optical element 215 rotates about a rotation axis 209, and the rotation speed of the second optical element 215 is different from that of the first optical element 214. The second optical element 215 is used to change the direction of the light beam projected by the first optical element 214. In one embodiment, the second optical element 115 is connected to another driver 217, and the driver 117 drives the second optical element 215 to rotate. The first optical element 214 and the second optical element 215 can be driven by the same or different drivers, so that the rotation speeds and / or rotation directions of the first optical element 214 and the second optical element 215 are different, thereby projecting the collimated light beam 219 to different directions in the external space, and a larger space range can be scanned. In one embodiment, the controller 218 controls the drivers 216 and 217 to drive the first optical element 214 and the second optical element 215 respectively. The rotation speeds of the first optical element 214 and the second optical element 215 can be determined according to the area and pattern expected to be scanned in actual applications. The drivers 216 and 217 can include motors or other drivers.

[0050] In one embodiment, the second optical element 115 includes a pair of opposite non-parallel surfaces through which the light beam passes. In one embodiment, the second optical element 115 includes a prism with a thickness varying along at least one radial direction. In one embodiment, the second optical element 115 includes a wedge-angle prism.

[0051] In one embodiment, the scanning module 102 further includes a third optical element (not shown in the figure) and a driver for driving the third optical element to move. Optionally, the third optical element includes a pair of opposite non-parallel surfaces through which the light beam passes. In one embodiment, the third optical element includes a prism with a thickness varying along at least one radial direction. In one embodiment, the third optical element includes a wedge-angle prism. At least two of the first, second, and third optical elements rotate at different rotation speeds and / or rotation directions.

[0052] The rotation of each optical element in the scanning module 202 can project light to different directions, such as directions 211 and 213, so as to scan the space around the lidar 200. As Figure 3 shown, Figure 3 is a schematic diagram of a scanning pattern of the lidar 200. It can be understood that when the speed of the optical element in the scanning module changes, the scanning pattern will also change accordingly.

[0053] When the light 211 projected by the scanning module 202 hits the detection object 201, a part of the light is reflected by the detection object 201 in the direction opposite to the projected light 211 and returns to the lidar 200. The returned light 212 reflected by the detection object 201 enters the collimating element 204 after passing through the scanning module 202.

[0054] The detector 205 and the emitter 203 are placed on the same side of the collimating element 204. The detector 205 is used to convert at least part of the return light passing through the collimating element 204 into an electrical signal.

[0055] In one embodiment, an anti-reflection film is coated on each optical element. Optionally, the thickness of the anti-reflection film is equal to or close to the wavelength of the light beam emitted by the emitter 103, which can increase the intensity of the transmitted light beam.

[0056] In one embodiment, a filter layer is coated on the surface of an element located on the light beam propagation path in the lidar, or a filter is provided on the light beam propagation path, which is used to transmit at least the light beam band emitted by the emitter and reflect other bands to reduce the noise brought by ambient light to the receiver.

[0057] In some embodiments, the emitter 203 may include a laser diode, and a nanosecond-level laser pulse is emitted through the laser diode. Further, the reception time of the laser pulse can be determined. For example, the reception time of the laser pulse is determined by detecting the rising edge time and / or the falling edge time of the electrical signal pulse. In this way, the lidar 200 can calculate the TOF using the pulse reception time information and the pulse emission time information, so as to determine the distance from the detected object 201 to the lidar 200.

[0058] In one implementation manner, the lidar according to the embodiment of the present application can be applied to a mobile platform, and the lidar can be installed on the platform body of the mobile platform. The mobile platform with the lidar can identify road markings. In certain implementation manners, the mobile platform includes at least one of an unmanned aerial vehicle, an automobile, a remote control vehicle, a robot, and a camera. When the lidar is applied to an unmanned aerial vehicle, the platform body is the fuselage of the unmanned aerial vehicle. When the lidar is applied to an automobile, the platform body is the body of the automobile. The automobile can be an autonomous vehicle or a semi-autonomous vehicle, which is not limited here. When the lidar is applied to a remote control vehicle, the platform body is the body of the remote control vehicle. When the lidar is applied to a robot, the platform body is the robot. When the lidar is applied to a camera, the platform body is the camera itself.

[0059] It should be noted that the type of the device including the road marking recognition device may not be limited in the embodiments of the present application. The device can be, for example, a server, an autonomous vehicle, a semi-autonomous vehicle, etc.

[0060] The road sign recognition method provided by the embodiment of the present application compresses the three-dimensional point cloud data obtained by lidar detection into a two-dimensional point cloud feature map, and processes the two-dimensional point cloud feature map to obtain the road sign recognition result. Since the lidar has high reliability and is hardly affected by environmental factors, even in a harsh imaging environment, the three-dimensional point cloud data obtained based on the lidar can obtain a relatively accurate road sign recognition result, so that the road sign recognition is no longer limited by the imaging environment, expanding the usage scenarios of road sign recognition, and solving the problem that the usage scenarios of the method for roadside sign recognition based on images in the traditional technology are relatively limited and difficult to meet the user requirements.

[0061] The following will describe in detail some embodiments of the present application with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0062] Figure 4 It is a schematic flowchart of the road sign recognition method provided by an embodiment of the present application. The execution subject of this embodiment can be a road sign recognition device, specifically, the processor of the road sign recognition device. As Figure 4 shown, the method of this embodiment may include:

[0063] Step 401: Obtain the three-dimensional point cloud data detected by the lidar, where the three-dimensional point cloud data includes the reflection data of the road sign area.

[0064] In this step, the lidar (Laser Radar) can also be referred to as a Light Detection and Ranging (LiDAR) system. The data obtained by scanning the surrounding environment with the lidar is the three-dimensional point cloud data. The three-dimensional point cloud data includes the reflection data of the road sign area. The reflection data can refer to the data carried by the reflected light after being reflected by the road sign area. It can be understood that the three-dimensional point cloud data is the data obtained by scanning the road with road signs by the lidar. Each point in the three-dimensional point cloud data can include three-dimensional coordinates and reflectivity information.

[0065] The road sign area can refer to the position range where the road sign is located. Among them, the road sign can specifically be any type of sign set on the road for marking driving specifications. Exemplarily, the road sign can be various lane lines, such as double solid lines, single solid lines, broken lines, or road diversion signs. The road sign can also include left-turn arrows, right-turn arrows, straight-ahead arrows, etc. It should be noted that the road signs that can be recognized by the road sign recognition method provided by the present application can be one or more of all road signs. The one or more road signs can be understood as specific road signs.

[0066] Step 402: Compress the three-dimensional point cloud data into a two-dimensional point cloud feature map.

[0067] In this step, the three-dimensional point cloud data in the three-dimensional space is compressed into a two-dimensional point cloud feature map in the two-dimensional space. Multiple points in the three-dimensional point cloud data can correspond to one pixel in the two-dimensional point cloud feature map. The two-dimensional point cloud feature map contains feature information that can be used to identify road markings.

[0068] It should be noted that the number of two-dimensional point cloud feature maps can correspond to the number of types of feature information. For example, assuming the number of types of feature information is 2, the number of two-dimensional point cloud feature maps can be 2, namely two-dimensional point cloud feature map 1 and two-dimensional point cloud feature Figure 2 , where the pixel value in the two-dimensional point cloud feature map 1 can represent one type of feature information, and the pixel value in the two-dimensional point cloud feature Figure 2 can represent another type of feature information.

[0069] Step 403: Process the two-dimensional point cloud feature map to obtain the road marking recognition result.

[0070] In this step, since the two-dimensional point cloud feature map contains feature information that can be used to identify roadside markings, the road marking recognition result can be obtained by processing the two-dimensional point cloud feature map. Exemplarily, the road marking recognition result can be whether a specific road marking is included in the three-dimensional point cloud data. For example, the road marking result can be that a single solid line is included or not included. Exemplarily, the road marking recognition result can be the category of the specific road marking included in the three-dimensional point cloud data. For example, the road marking result can be a single solid line or a broken line.

[0071] In this embodiment, by obtaining the three-dimensional point cloud data detected by the lidar, the three-dimensional point cloud data contains data of the road marking area, compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map, and processing the two-dimensional point cloud feature map to obtain the road marking recognition result. Since the lidar has high reliability and is very little affected by environmental factors, even in a harsh imaging environment, a relatively accurate road marking recognition result can be obtained based on the three-dimensional point cloud data obtained by the lidar, making the road marking recognition no longer limited by the imaging environment, expanding the usage scenario of the road marking recognition, and solving the problem that the usage scenario of the traditional method of identifying roadside markings based on images is relatively limited and difficult to meet the user's needs.

[0072] Figure 5 This is a schematic flowchart of the road marking recognition method provided by another embodiment of the present application. Based on the embodiment shown in Figure 5 , this embodiment mainly describes an optional implementation manner of processing the two-dimensional point cloud feature map to obtain the road marking recognition result. As shown in Figure 5 , the method of this embodiment may include:

[0073] Step 501: Obtain the three-dimensional point cloud data detected by the lidar. The three-dimensional point cloud data includes the reflection data of the road surface marking area.

[0074] It should be noted that Step 501 is similar to Step 401 and will not be elaborated here.

[0075] Step 502: Compress the three-dimensional point cloud data into a two-dimensional point cloud feature map.

[0076] In this step, compress the three-dimensional point cloud data according to the target direction to obtain a two-dimensional point cloud feature map containing feature information. The feature information includes relative height information and / or reflectivity information. The target direction is any direction from which feature information that can be used to identify road surface markings can be obtained. Exemplarily, the target direction includes the vertical direction. Correspondingly, the two-dimensional point cloud feature map can be understood as a two-dimensional horizontal plane point cloud feature map to facilitate implementation.

[0077] Among them, the relative height information may refer to the height information of an object relative to a target reference in the target direction. The target reference may include a lidar or a mobile platform for installing the lidar. Since road surface markings are set on the road surface, the relative height information between the road surface and the target reference usually meets certain conditions. Therefore, road surface markings can be identified according to the relative height information, that is, the relative height information can be used to identify road surface markings.

[0078] The reflectivity information may refer to the percentage of the echo energy collected by the lidar in the transmitted energy of the lidar. Since the reflectivity mainly depends on the nature of the object itself, as well as the incident wavelength and incident angle, when the incident wavelength and incident angle are fixed, objects can be identified according to the reflectivity, that is, the reflectivity information can be used to identify road surface markings.

[0079] By including relative height information in the feature information, the problem of information loss caused by compression can be avoided. By including both relative height information and reflectivity information in the feature information, it is beneficial to improve the recognition accuracy of road surface markings.

[0080] Exemplarily, the step of compressing the three-dimensional point cloud data according to the target direction to obtain a two-dimensional point cloud feature map containing reflectivity information and / or relative height information may specifically include the following Steps A and B.

[0081] Step A: Perform projection compression on the three-dimensional point cloud data along the target direction to obtain two-dimensional point cloud data.

[0082] Among them, each point in the two-dimensional point cloud data may include two-dimensional coordinates and reflectivity information. Optionally, the two-dimensional point cloud data may further include relative height information to avoid information loss problems caused by compression.

[0083] Step B: Extract feature information from the two-dimensional point cloud data to obtain a two-dimensional point cloud feature map containing the feature information.

[0084] Among them, by extracting feature information for each point in the two-dimensional point cloud data, a two-dimensional point cloud feature map containing the feature information can be obtained. The pixels in the two-dimensional point cloud feature map can correspond one-to-one with the points in the two-dimensional point cloud data.

[0085] Step 503: Output a road surface marking area based on the regional reflectivity and height of the two-dimensional point cloud feature map.

[0086] In this step, optionally, the road surface marking area can be used as the road surface marking recognition result.

[0087] Exemplarily, the two-dimensional point cloud feature map can be divided into multiple regions according to the reflectivity and height of each pixel in the two-dimensional point cloud feature map. Among them, one region can correspond to one object. Further, the road surface marking area can be determined from the multiple regions according to the reflectivity and height of the objects corresponding to each region in the multiple regions, as well as the target reflectivity and target height. The target reflectivity can represent the reflectivity when the object is a road surface marking, and the target height can represent the height rate when the object is a road surface marking.

[0088] Alternatively, optionally, the road surface marking recognition result can be further determined according to the road surface marking area. Exemplarily, it can be further determined what specific road surface marking the road surface marking area corresponds to. Exemplarily, after step 503, it may further include: performing clustering processing on the output road surface marking area; identifying the result of the clustering processing and outputting the road surface marking corresponding to the result. Exemplarily, the road surface marking area output in step 502 can be clustered by a clustering algorithm to divide the road surface marking areas corresponding to the same road surface marking into one cluster and divide the road surface marking areas corresponding to different road surface markings into different clusters. For example, assuming that the road surface marking area includes a road surface marking area a corresponding to a solid line, a road surface marking area b corresponding to a broken line, a road surface marking area c corresponding to a solid line, and a road surface marking area d corresponding to a broken line, then through clustering processing, the clustering processing result can be obtained that divides the road surface marking areas a and b into one cluster and divides the road surface marking areas b and d into another cluster.

[0089] By first performing clustering and then identifying based on the results of the clustering process, since clustering can divide the road surface marking areas corresponding to the same road surface marking into a clustering cluster, when identifying based on the results of the clustering process, it is possible to determine the category of the road surface marking based on all the road surface marking areas of one road surface marking, which is beneficial to improving the accuracy of identification.

[0090] Exemplarily, the identifying the results of the clustering process and outputting the road surface marking corresponding to the results may specifically include: performing pixel-level comparison on the two-dimensional point cloud feature map according to the results to obtain the road surface marking corresponding to the results. Exemplarily, the arrangement of the pixels belonging to the clustering cluster in the two-dimensional point cloud feature map can be determined according to the road surface marking areas in a single clustering cluster, and the road surface marking corresponding to the clustering cluster can be determined according to this arrangement. For example, assuming that the arrangement of the pixels belonging to a clustering cluster in the two-dimensional point cloud feature map is non-continuous arrangement along a straight line, then the road surface marking corresponding to the clustering cluster is a discontinuous line. By performing pixel-level comparison on the two-dimensional point cloud feature map according to the results, it is beneficial to improve the confidence of the obtained road surface marking.

[0091] In this embodiment, by obtaining the three-dimensional point cloud data detected by the lidar, the three-dimensional point cloud data includes the data of the road surface marking areas, compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map, and outputting the road surface marking areas based on the regional reflectivity and height of the two-dimensional point cloud feature map, a road surface marking recognition method based on the regional reflectivity and height of the two-dimensional point cloud feature map is realized, which is beneficial to simplifying the implementation.

[0092] Figure 6 For the flow schematic diagram of the road surface marking recognition method provided in another embodiment of the present application, this embodiment mainly describes another optional implementation manner of processing the two-dimensional point cloud feature map to obtain the road surface marking recognition result on the basis of the embodiment shown in Figure 6 As shown in Figure 6 The method of this embodiment may include:

[0093] Step 601, obtaining the three-dimensional point cloud data detected by the lidar, where the three-dimensional point cloud data includes the data of the road surface marking areas.

[0094] It should be noted that step 601 is similar to step 401 and will not be elaborated here.

[0095] Step 602, compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map.

[0096] It should be noted that step 602 is similar to step 401 and step 501 and will not be elaborated here.

[0097] Step 603: Input the two-dimensional point cloud feature map into a preset neural network model to obtain the model output result of the preset neural network model.

[0098] In this step, the preset neural network model is used to determine the surface object category of each pixel in the two-dimensional point cloud feature map. The preset neural network model may include multiple output channels, and the multiple output channels correspond one-to-one to multiple surface object categories, and the multiple surface object categories include at least one road marking category. The output channel is used to output a confidence feature map of the corresponding surface object category, and the confidence feature map is used to characterize the probability that a pixel is the corresponding surface object category. Among them, the pixels in the confidence feature map can correspond one-to-one to the pixels in the two-dimensional point cloud feature map.

[0099] For example, assume that the number of road marking categories is 3, namely single solid line, broken line, and left turn arrow, and the output channel corresponding to the single solid line outputs confidence feature map 1, the output channel corresponding to the broken line outputs confidence feature Figure 2 , the output channel corresponding to the left turn arrow outputs confidence feature Figure 3 , then the pixel value in confidence feature map 1 can characterize the probability that a pixel is a single solid line, and the confidence feature Figure 2 The pixel value in can characterize the probability that a pixel is a broken line, and the confidence feature Figure 3 The pixel value in can characterize the probability that a pixel is a left turn arrow. It should be noted that in the embodiments of the present application, a pixel being a surface object category can be understood as the pixel position of the pixel being the recognized surface object category.

[0100] It should be noted that when the multiple surface object categories include a road marking category, the preset neural network can be used to identify a road marking category, and the road marking category can represent a specific road marking category, or can represent a set of at least two road marking categories.

[0101] Exemplarily, the multiple surface object categories may further include other surface object categories. Correspondingly, the model output result may further include confidence feature maps of other surface object categories. For example, the model output result may further include a confidence feature map of a building, and the pixel value in the confidence feature map can characterize the probability that a pixel is a building. For example, the model output result may further include a confidence feature map of a building, and the pixel value in the confidence feature map can characterize the probability that a pixel is a building.

[0102] Exemplarily, the multiple surface object categories may further include "other" for representing surface objects for which the category cannot be recognized, to distinguish from surface objects for which the preset neural network model can recognize the category.

[0103] Exemplarily, the output result of the model can be used as the road surface marking recognition result, thus simplifying the implementation of the road surface marking recognition method.

[0104] Alternatively, exemplarily, after step 603, it may further include: obtaining the road surface marking recognition result based on the output result of the model. Thereby, a more specific road surface marking recognition result can be obtained, which is beneficial for subsequent processing after obtaining the road surface marking recognition method.

[0105] Exemplarily, the obtaining the road surface marking recognition result according to the output result of the model includes: taking the surface object category corresponding to the confidence feature map with the largest pixel value at the same pixel position in the confidence feature maps of the multiple surface object categories as the surface object category at the pixel position.

[0106] Assume that the number of output channels of the preset neural network model is 4, and the four confidence feature maps are confidence feature map 1 to confidence feature Figure 4 , and confidence feature map 1 corresponds to a single solid line, confidence feature Figure 2 corresponds to a broken line, confidence feature Figure 3 corresponds to a left turn arrow, confidence feature Figure 4 corresponds to "other". For example, when the pixel value at pixel position (100, 100) in confidence feature map 1 is 70, confidence feature Figure 2 the pixel value at pixel position (100, 100) is 50, confidence feature Figure 3 the pixel value at pixel position (100, 100) is 20, confidence feature Figure 4 the pixel value at pixel position (100, 100) is 20, it can be determined that the surface object category at pixel position (100, 100) is a single solid line. Another example, when the pixel value at pixel position (100, 80) in confidence feature map 1 is 20, confidence feature Figure 2 the pixel value at pixel position (100, 80) is 30, confidence feature Figure 3 the pixel value at pixel position (100, 80) is 20, confidence feature Figure 4 the pixel value at pixel position (100, 80) is 70, it can be determined that the surface object category at pixel position (100, 80) is "other", that is, not any one of a single solid line, a broken line, and a left turn arrow.

[0107] Since the pixel positions in the confidence feature map and the pixel positions in the two-dimensional point cloud feature map can be in one-to-one correspondence, the surface object category at the above pixel positions can represent the surface object category at the pixel positions in the two-dimensional point cloud feature map.

[0108] Exemplarily, the surface object category at the pixel position in the two-dimensional point cloud feature map can be used as the road surface marking recognition result.

[0109] Alternatively, exemplarily, the method of this embodiment may further include using the surface object category at the pixel position as the surface object category of the point corresponding to the pixel position in the three-dimensional point cloud data, that is, the surface object category of the point in the three-dimensional point cloud data can be used as the road surface marking recognition result.

[0110] Exemplarily, the preset neural network model may specifically be a Convolutional Neural Networks (CNN) model. The structure of the preset neural network model can be, for example, as Figure 7 shown. As Figure 7 shown, the preset neural network model may include multiple computing nodes. Each computing node may include a Convolution (Conv) layer, Batch Normalization (BN), and the activation function ReLU. The computing nodes may be connected in a Skip Connection manner. Input data of K×H×W can be input into the preset neural network model. After being processed by the preset neural network model, output data of C×H×W can be obtained. Among them, K can represent the number of two-dimensional point cloud feature maps, H can represent the height of the two-dimensional point cloud feature map, W can represent the width of the two-dimensional point cloud feature map, and C can represent the number of categories.

[0111] It should be noted that when the two-dimensional point cloud feature map is too large, a two-dimensional point cloud feature map can be cut into N sub-feature maps. Correspondingly, the input data can be N×K×H’×W’, and the output data can be N×C×H’×W’. Among them, H’ can represent the height of the sub-feature map, and W’ can represent the width of the sub-feature map.

[0112] In this embodiment, by obtaining the three-dimensional point cloud data detected by the lidar, the three-dimensional point cloud data contains data of the road surface marking area, compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map, inputting the two-dimensional point cloud feature map into the preset neural network model, obtaining the model output result of the preset neural network model, and distinguishing the semantics in the two-dimensional point cloud feature map based on the preset neural network model to obtain the road surface marking recognition result, the road surface marking recognition result is obtained through the preset neural network model.

[0113] Optionally, based on the above embodiments, the compression of the three-dimensional point cloud data into a two-dimensional point cloud feature map may specifically include: screening the three-dimensional point cloud data according to a screening condition to obtain the screened three-dimensional point cloud data, and compressing the screened three-dimensional point cloud data into a two-dimensional point cloud feature map. By filtering out irrelevant data that does not need to be concerned about in advance before processing the two-dimensional point cloud feature map, the data volume of the two-dimensional point cloud feature map can be reduced, and the influence of irrelevant data on the accuracy of road sign recognition results can be avoided. Exemplarily, the screening condition may include a distance condition and / or a height condition. Among them, the distance condition may be, for example, a distance less than 100 meters, so as to screen out the three-dimensional point cloud data that meets the condition of being less than 100 meters away from the lidar from the three-dimensional point cloud data, thereby avoiding unnecessary recognition of points with too far distances and being beneficial to saving computing resources. The height condition may be, for example, a relative height greater than height threshold 1 and less than height threshold 2, where height threshold 1 is less than height threshold 2, so as to screen out the three-dimensional point cloud data that meets the condition of having a relative height greater than height threshold 1 and less than height threshold 2 from the three-dimensional point cloud data, thereby avoiding unnecessary recognition of points that are not likely to be road signs and being beneficial to saving computing resources.

[0114] Optionally, based on the above embodiments, the obtaining of the three-dimensional point cloud data detected by the lidar may specifically include: obtaining multiple frames of three-dimensional point cloud data detected by the lidar; accumulating the multiple frames of three-dimensional point cloud data to obtain the accumulated three-dimensional point cloud data. Through the method of multi-frame accumulation, a denser point cloud can be obtained, avoiding the problem of sparse point clouds due to a small number of points detected in a single frame. Correspondingly, the compression of the three-dimensional point cloud data into a two-dimensional point cloud feature map may specifically include: compressing the accumulated three-dimensional point cloud data into a two-dimensional point cloud feature map.

[0115] Optionally, based on the above method embodiments, the following steps C and D may further be included.

[0116] Step C, determining whether a preset environmental condition is met.

[0117] Step D, if the preset environmental condition is met, obtaining the image information collected by the image acquisition module, where the image information includes the road sign area, and processing the image information to obtain a road sign recognition result.

[0118] Among them, similar to the above lidar, the road sign recognition device may include the image acquisition module. Specifically, obtaining the image information in step D may include obtaining the image information through image acquisition by the road sign recognition device; alternatively, the road sign recognition device may not include the image acquisition module, and specifically, obtaining the image information in step D may include receiving the image information collected by the image acquisition module sent by other devices. Among them, the image acquisition module may be, for example, a camera.

[0119] The preset environmental conditions may refer to the imaging environmental conditions that need to be met when using the method of obtaining the road sign recognition result based on the image information. When the preset environmental conditions are met, it may indicate that the imaging environment is good, and the imaging quality of the obtained image information is good. Therefore, it can be considered that the accuracy of the road sign recognition result obtained based on the image information is relatively high, and thus the method of obtaining the road sign recognition result based on the image information can be adopted. When the preset environmental conditions are not met, it may indicate that the imaging environment is poor, and the imaging quality of the obtained image information is poor. Therefore, it can be considered that the accuracy of the road sign recognition result obtained based on the image information is relatively low, and thus the method of obtaining the road sign recognition result based on the image information may not be adopted.

[0120] The preset environmental conditions may include one or more of any environmental factors that affect the imaging quality. Exemplarily, the preset environmental conditions may include environmental light conditions and / or lens dirtiness conditions. Exemplarily, the environmental light conditions may include that the environmental light intensity is greater than the intensity threshold. Exemplarily, the lens dirtiness condition includes that the lens dirtiness degree is less than the degree threshold.

[0121] It should be noted that the specific method for processing the image information to obtain the road sign recognition result is not limited in the embodiments of the present application.

[0122] Through step C and step D, the road sign recognition device can not only support the function of road sign recognition based on three-dimensional point cloud data, but also support the function of road sign recognition based on image information, improving the flexibility of the road sign recognition device in recognizing road signs.

[0123] Exemplarily, the road sign recognition result obtained by processing the two-dimensional point cloud feature map (hereinafter referred to as the first road sign recognition result) and the road sign recognition result obtained by processing the image information (hereinafter referred to as the second road sign recognition result) can be listed as the final road sign recognition result. For example, the first road sign recognition result can be used as the input of one function, and the second road sign result can be used as the input of another function.

[0124] Alternatively, exemplarily, according to the fusion strategy, the first road surface identification result and the second road surface identification result can be fused to obtain the fused road surface identification result, that is, the fused road surface identification result can be used as the final road surface identification result. By fusing the first road surface identification result and the second road surface identification result, the identification effect of the road surface identification can be improved.

[0125] Exemplarily, the fusion strategy may include a fusion strategy based on the distance. Wherein, this distance may refer to the road surface distance.

[0126] Exemplarily, the step of fusing the first road surface identification result and the second road surface result according to the fusion strategy to obtain the fused road surface identification result may specifically include: for the visual field range with a distance greater than the distance threshold, taking the first road surface identification result as the fused road surface identification result. Since the larger the distance, the larger the actual area corresponding to a single pixel in the image information, and the worse the accuracy of the distance determined according to the image information, taking the first road surface identification result as the fused road surface identification result for the visual field range with a distance greater than the distance threshold can avoid the problem of inaccurate identification results caused by taking the second road surface identification result as the road surface identification result for the visual field range with a distance greater than the distance threshold.

[0127] And / or, exemplarily, the step of fusing the first road surface identification result and the second road surface result according to the fusion strategy to obtain the fused road surface identification result may specifically include: for the visual field range with a distance less than the distance threshold, taking the second road surface identification result as the fused road surface identification result. Since the cost of the lidar is much higher than that of the image acquisition module, taking the second road surface identification result as the fused road surface identification result for the visual field range with a distance less than the distance threshold is beneficial to cost savings.

[0128] Alternatively, exemplarily, the method of obtaining the road surface identification result based on the image information may be selected when the imaging environment is good, and the method of obtaining the road surface identification result based on the three-dimensional point cloud data may be selected when the imaging environment is poor. Exemplarily, the above-mentioned obtaining the three-dimensional point cloud data detected by the lidar includes: the step of obtaining the three-dimensional point cloud data detected by the lidar if the preset environmental conditions are not met.

[0129] Optionally, based on the above embodiments, it may further include: displaying the recognition result of the road surface markings. This is beneficial for the user to view the recognition result of the road surface markings. Exemplarily, the displaying the recognition result of the road surface markings includes: marking the road surface markings in the target image according to the recognition result of the road surface markings to obtain a marked image, and displaying the marked image. Exemplarily, different colors can be used to mark different categories of road surface markings. For example, green represents a solid single line, yellow represents a broken line, and purple represents an edge line.

[0130] Exemplarily, the target image includes one or more of the following: a completely black image, a completely white image, or an image including the road surface marking area. Among them, a completely black image can be an image in which the red (R), green (G), and blue (B) values of each pixel are all 0, and a completely white image can be an image in which the R, G, and B values of each pixel are all 255.

[0131] Optionally, based on the above embodiments, if the road surface is flat, when the target direction is the vertical direction, the road surface distance between the lidar and the road surface markings obtained according to the compressed three-dimensional point cloud data (i.e., the two-dimensional point cloud feature map) can match the actual road surface distance between the lidar and the road surface markings. Therefore, by setting the target direction to the vertical direction, the accuracy of the road surface distance between the obtained road surface markings and the lidar can be ensured when the road surface is flat.

[0132] However, for a scenario where the road surface is uneven, if the target direction is the vertical direction, there will be a situation where the road surface distance between the lidar and the road surface markings obtained according to the compressed three-dimensional point cloud data does not match the actual road surface distance between the lidar and the road surface markings. Or rather, in the case of an uneven road surface, the length of the road surface markings obtained according to the compressed three-dimensional point cloud data, such as the length of the lane line, will be shorter than its actual length. The same problem exists for other road surface markings, which will cause the road surface marking information obtained through the lidar point cloud to not correspond to the road surface marking information obtained by other sensors. For the road surface markings that need to be recognized, due to the compression of the shape of the road surface markings, there may also be a problem of misrecognition.

[0133] As Figure 8 shown, for the flat road surface R1, the road surface distance L1 between the lidar O and the road surface marking A1 obtained according to the three-dimensional point cloud data compressed based on the vertical direction d1 is the actual road surface distance between the lidar O and the road surface marking A1. For the uphill road surface R2, the road surface distance L2 between the lidar O and the road surface marking A2 obtained according to the three-dimensional point cloud data compressed based on the vertical direction d1 is less than the actual road surface distance L3 + L4 between the lidar O and the road surface marking A2. It should be noted that Figure 8Taking the flat road surface as an example, for the scenario where the road surface is curved, when the target direction is the vertical direction, there is also the problem that the road surface distance between the lidar and the road markings is inaccurate.

[0134] Based on Figure 8 , assuming that the road marking A2 is a single solid line, the relationship between the actual single solid line 901 and the single solid line 902 identified from the compressed three-dimensional point cloud data can be as Figure 9A shown. It can be seen that the length of the identified single solid line 902 is shorter than the length of the actual single straight line 901.

[0135] Based on Figure 8 , assuming that the road marking A2 is an arrow, the relationship between the actual arrow 903 and the arrow 904 identified from the compressed three-dimensional point cloud data can be as Figure 9B shown. It can be seen that the length of the identified arrow 904 is shorter than the length of the actual arrow 903, and the triangle of the identified arrow 904 is flatter than the triangle of the actual arrow 903.

[0136] Therefore, in order to avoid the problem of inaccurate road surface distance between the lidar and the road markings, the target direction can be dynamically determined according to the undulating state of the road surface, so that the road surface distance between the lidar and the road markings obtained from the three-dimensional point cloud data compressed based on the target direction can match the actual road surface distance.

[0137] Exemplarily, the undulating state of the road surface can be obtained by means of three-dimensional modeling according to the three-dimensional point cloud data detected by the lidar, and the three-dimensional point cloud data can be compressed according to the undulating state of the road surface. Exemplarily, the object relationship between different point cloud ranges and the target direction can be determined according to the undulating state of the road surface, and the point cloud data within the corresponding point cloud range can be compressed according to the target direction. For example, as Figure 10 shown, the target direction corresponding to the three-dimensional point cloud data within the flat road surface range can be determined as the vertical direction d1 according to the road surface platform state, and the target direction corresponding to the three-dimensional point cloud data within the uphill road surface range can be determined as the inclined direction d2 perpendicular to the road surface. Further, the three-dimensional point cloud data within the flat road surface range can be compressed according to the vertical direction d1, and the three-dimensional point cloud data within the uphill road surface range can be compressed according to the inclined direction d2. And, by using Figure 10 this method, the road surface distance L1 between the lidar and the road marking A1 obtained from the compressed three-dimensional point cloud data is the actual road surface distance between the lidar and the road marking, and the road surface distance L3 + L4 between the lidar and the road marking A2 obtained from the compressed or three-dimensional point cloud data is the actual road surface distance between the lidar and the road marking. At the same time, based on the method of this embodiment, the problem that the vehicle identifies the change in the size or shape of the road marking can be solved.

[0138] It should be noted that Figure 10 in this case, the flat road surface is taken as an example. For the scenario where the road surface is a curved surface, the road surface can be divided into multiple road surface ranges according to a certain granularity. For each road surface range among the multiple road surface ranges, the direction perpendicular to its tangent plane can be used as the target direction corresponding to the three-dimensional point cloud data within its range. It can be understood that the smaller the granularity, the higher the accuracy of the road surface distance between the lidar and the road surface markings.

[0139] For a curved road surface, such as Figure 11 as shown, the road surface in front of the vehicle can be divided into multiple road surface ranges, and the three-dimensional point cloud data of each road surface range can be compressed according to the corresponding target direction. It should be noted that Figure 11 in this case, taking 13 road surface ranges as an example, the arrow above a road surface range can represent the target direction corresponding to this road surface range.

[0140] Figure 12 is a schematic structural diagram of a road surface marking recognition device provided by an embodiment of the present application. As Figure 12 shown, the device 1200 may include: a processor 1201 and a memory 1202.

[0141] The memory 1202 is used to store program codes;

[0142] The processor 1201 calls the program codes, and when the program codes are executed, it is used to perform the following operations:

[0143] Obtain the three-dimensional point cloud data detected by the lidar, and the three-dimensional point cloud data includes the reflection data of the road surface marking area;

[0144] Compress the three-dimensional point cloud data into a two-dimensional point cloud feature map;

[0145] Process the two-dimensional point cloud feature map to obtain a road surface marking recognition result.

[0146] The road surface marking recognition device provided by this embodiment can be used to execute the technical solutions of the foregoing method embodiments. Its implementation principle and technical effects are similar to those of the method embodiments, and will not be elaborated here.

[0147] Those of ordinary skill in the art can understand that all or part of the steps to implement the foregoing method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the foregoing method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disc that can store program codes.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying road surface markings, characterized in that, it includes: Obtain the three-dimensional point cloud data detected by a lidar, and the three-dimensional point cloud data includes the reflection data of the road surface marking area; Compress the three-dimensional point cloud data into a two-dimensional point cloud feature map; wherein, the two-dimensional point cloud feature map includes relative height information and reflectivity information; wherein, the relative height information is used to identify road surface markings; Based on the regional reflectivity and height of the two-dimensional point cloud feature map, output the road surface marking area; Perform clustering processing on the output road surface marking area; Perform pixel-level comparison on the two-dimensional point cloud feature map according to the result of the clustering processing to obtain the road surface marking corresponding to the result.

2. The method according to claim 1, characterized in that, The step of compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map includes: Compress the three-dimensional point cloud data according to a target direction to obtain a two-dimensional point cloud feature map containing feature information, and the target direction is a direction that can retain the feature information for identifying road surface markings.

3. The method according to claim 2, characterized in that, Compressing the three-dimensional point cloud data according to a target direction to obtain a two-dimensional point cloud feature map containing reflectivity information and relative height information includes: Perform projection compression on the three-dimensional point cloud data along the target direction to obtain two-dimensional point cloud data; Extract feature information from the two-dimensional point cloud data to obtain a two-dimensional point cloud feature map containing the feature information.

4. The method according to any one of claims 1-3, characterized in that, The step of compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map includes: According to a screening condition, screen the three-dimensional point cloud data to obtain the screened three-dimensional point cloud data; Compress the screened three-dimensional point cloud data into a two-dimensional point cloud feature map.

5. The method according to claim 4, characterized in that, The screening condition includes a distance condition and / or a height condition.

6. The method according to any one of claims 1-3, characterized in that, The step of obtaining the three-dimensional point cloud data detected by a lidar includes: Obtain multiple frames of three-dimensional point cloud data detected by a lidar; Accumulate the multiple frames of three-dimensional point cloud data to obtain the accumulated three-dimensional point cloud data; The step of compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map includes: Compress the accumulated three-dimensional point cloud data into a two-dimensional point cloud feature map.

7. The method according to any one of claims 1-3, characterized in that, The method further includes: Determine whether a preset environmental condition is satisfied; If the preset environmental condition is satisfied, obtain the image information collected by an image acquisition module, the image information includes the road surface marking area, and process the image information to obtain a road surface marking recognition result.

8. The method according to claim 7, characterized in that, The method further includes: According to a fusion strategy, fuse the first road surface marking recognition result obtained by processing the two-dimensional point cloud feature map and the second road surface marking recognition result obtained by processing the image information to obtain a fused road surface marking recognition result.

9. The method according to claim 8, characterized in that, The fusion strategy includes a fusion strategy based on distance.

10. The method according to claim 9, wherein, according to the fusion strategy, fusing the first road surface identification result and the second road surface identification result to obtain a fused road surface identification result, including: For a visual field range where the distance is greater than the distance threshold, using the first road surface identification result as the fused road surface identification result.

11. The method according to claim 9, wherein, according to the fusion strategy, fusing the first road surface identification result and the second road surface identification result to obtain a fused road surface identification result, including: For a visual field range where the distance is less than the distance threshold, using the second road surface identification result as the fused road surface identification result.

12. The method according to claim 7, wherein, obtaining the three-dimensional point cloud data detected by the lidar includes: If the preset environmental conditions are not met, performing the step of obtaining the three-dimensional point cloud data detected by the lidar.

13. The method according to claim 7, wherein, the preset environmental conditions include environmental light conditions and / or lens dirtiness degree conditions.

14. The method according to claim 13, wherein, the environmental light conditions include that the environmental light intensity is greater than the intensity threshold.

15. The method according to claim 13, wherein, the lens dirtiness degree conditions include that the lens dirtiness degree is greater than the degree threshold.

16. The method according to any one of claims 1-3, wherein, the method further includes: displaying the road surface identification result.

17. The method according to claim 16, wherein, displaying the road surface identification result includes: Annotating the road surface identification on the target image according to the road surface identification result to obtain an annotated image, and displaying the annotated image.

18. The method according to claim 17, wherein, the target image includes one or more of the following: a completely black image, a completely white image, or an image containing the road surface identification area.

19. The method according to any one of claims 1-3, wherein, the lidar is disposed on a mobile platform.

20. The method according to claim 19, wherein, the mobile platform includes an autonomous vehicle and / or a semi-autonomous vehicle.

21. A road surface identification device, wherein, including: a processor and a memory; the memory is used for storing program codes; the processor, when calling the program codes and the program codes are executed, is used for performing the following operations: Obtaining the three-dimensional point cloud data detected by the lidar, where the three-dimensional point cloud data includes reflection data of the road surface identification area; Compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map; wherein, the two-dimensional point cloud feature map includes relative height information and reflectivity information; wherein, the relative height information is used for identifying road surface identifications; Outputting the road surface identification area based on the regional reflectivity and height of the two-dimensional point cloud feature map. Perform clustering processing on the output road surface marking area; Perform pixel-level comparison on the two-dimensional point cloud feature map according to the result of the clustering processing to obtain the road surface marking corresponding to the result.

22. The device according to claim 21, wherein, The processor is used to compress the three-dimensional point cloud data into a two-dimensional point cloud feature map, specifically including: Compress the three-dimensional point cloud data according to the target direction to obtain a two-dimensional point cloud feature map containing feature information, and the target direction is the direction that can retain the feature information for identifying road surface markings.

23. The device according to claim 22, wherein, The processor is used to compress the three-dimensional point cloud data according to the target direction to obtain a two-dimensional point cloud feature map containing reflectivity information and relative height information, specifically including: Perform projection compression on the three-dimensional point cloud data along the target direction to obtain two-dimensional point cloud data; Extract feature information from the two-dimensional point cloud data to obtain a two-dimensional point cloud feature map containing the feature information.

24. The device according to any one of claims 21-23, wherein, The processor is used to compress the three-dimensional point cloud data into a two-dimensional point cloud feature map, specifically including: Filter the three-dimensional point cloud data according to the filtering conditions to obtain the filtered three-dimensional point cloud data; Compress the filtered three-dimensional point cloud data into a two-dimensional point cloud feature map.

25. The device according to claim 24, wherein, The filtering conditions include distance conditions and / or height conditions.

26. The device according to any one of claims 21-23, wherein, The processor is used to obtain the three-dimensional point cloud data detected by the lidar, specifically including: Obtain multiple frames of three-dimensional point cloud data detected by the lidar; Accumulate the multiple frames of three-dimensional point cloud data to obtain the accumulated three-dimensional point cloud data; The step of compressing the three-dimensional point cloud data into a two-dimensional point cloud feature map includes: Compress the accumulated three-dimensional point cloud data into a two-dimensional point cloud feature map.

27. The device according to any one of claims 21-23, wherein, The processor is further used for: Determine whether the preset environmental conditions are met; If the preset environmental conditions are met, obtain the image information collected by the image acquisition module, the image information includes the road surface marking area, and process the image information to obtain the road surface marking recognition result.

28. The device according to claim 27, wherein, The processor is further used for: Fuse the first road surface marking recognition result obtained by processing the two-dimensional point cloud feature map and the second road surface marking recognition result obtained by processing the image information according to the fusion strategy to obtain the fused road surface marking recognition result.

29. The device according to claim 28, wherein, The fusion strategy includes a fusion strategy based on distance.

30. The device according to claim 29, wherein, The processor is used to fuse the first road surface marking recognition result and the second road surface marking recognition result according to the fusion strategy to obtain the fused road surface marking recognition result, specifically including: For the field of view with a distance greater than the distance threshold, use the first road surface identification result as the fused road surface identification result.

31. The device according to claim 29, wherein, the processor is configured to fuse the first road surface identification result and the second road surface identification result according to a fusion strategy to obtain a fused road surface identification result, specifically including: For the field of view with a distance less than the distance threshold, use the second road surface identification result as the fused road surface identification result.

32. The device according to claim 27, wherein, the processor is configured to obtain three-dimensional point cloud data detected by a lidar, specifically including: If the preset environmental conditions are not met, perform the step of obtaining three-dimensional point cloud data detected by the lidar.

33. The device according to claim 27, wherein, the preset environmental conditions include environmental light conditions and / or lens dirtiness conditions.

34. The device according to claim 33, wherein, the environmental light conditions include that the environmental light intensity is greater than an intensity threshold.

35. The device according to claim 33, wherein, the lens dirtiness conditions include that the lens dirtiness degree is greater than a degree threshold.

36. The device according to any one of claims 21-23, wherein, the processor is further configured to: display the road surface identification result.

37. The device according to claim 36, wherein, the processor is configured to display the road surface identification result, specifically including: Mark the road surface identification result in the target image to obtain a marked image, and display the marked image.

38. The device according to claim 37, wherein, the target image includes one or more of the following: a completely black image, a completely white image, or an image including the road surface identification area.

39. The device according to any one of claims 21-23, wherein, the lidar is arranged on a mobile platform.

40. The device according to claim 39, wherein, the mobile platform includes an autonomous vehicle and / or a semi-autonomous vehicle.

41. A computer-readable storage medium, wherein, the computer-readable storage medium stores a computer program, the computer program includes at least one segment of code, and the at least one segment of code can be executed by a computer to control the computer to execute the method according to any one of claims 1-20.

42. A computer program product, including a computer program, wherein, when the computer program is executed by a computer, it is used to implement the method according to any one of claims 1-20.

Images