Method, device and computer readable storage medium for identifying noise under a sign

Abstract

The application provides a method, device and computer readable storage medium for identifying noise points under a road sign. The method comprises: acquiring image data and point cloud data of a region in front of a target vehicle, the region in front comprising a road sign; processing the image data using a 3D camera model to obtain a first processing result; processing the point cloud data using a laser radar model to obtain a second processing result; clustering the point cloud data to obtain a clustering result; and determining whether noise points in a region under the road sign are real objects according to an intersection of the first processing result and the clustering result, and / or according to an intersection of the second processing result and the clustering result. The present scheme determines whether noise points in a region under a road sign are real objects, thereby further ensuring the accuracy of vehicle path planning.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, in particular, to a method and device for identifying noise points under a road sign, a computer readable storage medium, a processor and a vehicle. BACKGROUND

[0002] In the field of automatic driving and assisted driving, in order to achieve accurate planning of the path of a vehicle, it is often necessary to identify road signs above the road and on both sides of the road. In rainy, foggy and other weather conditions, noise points under the road sign are often identified as real objects, which affects the accuracy of the vehicle path planning.

[0003] The existing solutions lack a method for accurately identifying whether noise points under a road sign are real objects. SUMMARY

[0004] The main purpose of the present application is to provide a method and device for identifying noise points under a road sign, a computer readable storage medium, a processor and a vehicle, to at least solve the problem that the existing solutions lack a method for accurately identifying whether noise points under a road sign are real objects.

[0005] In order to achieve the above purpose, according to one aspect of the present application, a method for identifying noise points under a road sign is provided, comprising: acquiring image data and point cloud data of a region in front of a target vehicle, the region in front including a road sign; processing the image data using a 3D camera model to obtain a first processing result; processing the point cloud data using a laser radar model to obtain a second processing result; clustering the point cloud data to obtain a clustering result; determining whether noise points in a region under the road sign are real objects according to the intersection of the first processing result and the clustering result, and / or according to the intersection of the second processing result and the clustering result.

[0006] Further, the method for processing the image data using a 3D camera model to obtain a first processing result comprises: processing the image data using a 3D camera model to obtain a first bounding box; scaling the first bounding box along the driving direction of the target vehicle according to a scaling ratio to obtain a second bounding box, the scaling ratio being determined by the distance between the target vehicle and a target object in front.

[0007] Further, the second processing result is a third bounding box, the clustering result is an outer bounding box of a shape obtained by clustering, and whether the noise point in the area below the signboard is a real object is determined according to an intersection of the first processing result and the clustering result, and / or according to an intersection of the second processing result and the clustering result, including: obtaining a first proportion of a volume of an overlapping part of the second bounding box and the outer bounding box to a total volume of the outer bounding box; obtaining a second proportion of a volume of an overlapping part of the third bounding box and the outer bounding box to the total volume of the outer bounding box; in a case where the first proportion is greater than a predetermined proportion, and / or the second proportion is greater than the predetermined proportion, determining that the noise point in the area below the signboard is a real object; otherwise, determining that the noise point in the area below the signboard is not a real object.

[0008] Further, after determining that the noise point in the area below the signboard is not a real object, the method further includes: deleting the noise point in the area below the signboard.

[0009] Further, according to the first processing result, the second processing result and the clustering result, whether the noise point in the area below the signboard is a real object is determined, including: obtaining a distance between the target vehicle and the signboard; in a case where the distance between the target vehicle and the signboard is greater than or equal to a first distance and less than or equal to a second distance, and the second distance is greater than the first distance, determining whether the noise point in the area below the signboard is a real object according to an intersection of the first processing result and the clustering result, and / or according to an intersection of the second processing result and the clustering result.

[0010] Further, the method further includes: in a case where the distance between the target vehicle and the signboard is less than the first distance, determining that the noise point in the area below the signboard is a real object.

[0011] Further, the method further includes: in a case where the distance between the target vehicle and the signboard is greater than a second distance, retaining the noise point in the area below the signboard.

[0012] Further, the method further includes: removing a predetermined width on both sides along a horizontal axis direction of the image data to obtain a cropped image; obtaining a first width of a sub-image corresponding to the front target object in the image data; obtaining a second width of the sub-image corresponding to the front target object in the cropped image; in a case where the first width is greater than the second width, determining that a probability that the image data does not cover the whole of the front target object is greater than a predetermined probability.

[0013] Further, the method further includes: obtaining reflectivity of each point in the point cloud data; and retaining points with reflectivity greater than a predetermined reflectivity.

[0014] Further, the target vehicle travels on a highway.

[0015] Further, the 3D camera model is a first deep learning model trained by using a first convolutional neural network, and the LiDAR model is a second deep learning model trained by using a second convolutional neural network.

[0016] According to another aspect of the present application, there is provided an apparatus for identifying a noise point under a road sign, comprising: a first obtaining unit configured to obtain image data and point cloud data of a region in front of a target vehicle, the region in front comprising a road sign; a first processing unit configured to process the image data by using a 3D camera model to obtain a first processing result; a second processing unit configured to process the point cloud data by using a LiDAR model to obtain a second processing result; a clustering unit configured to cluster the point cloud data to obtain a clustering result; and a first determining unit configured to determine whether a noise point in a region under the road sign is a real object according to an intersection of the first processing result and the clustering result, and / or according to an intersection of the second processing result and the clustering result.

[0017] According to yet another aspect of the present application, there is provided a computer-readable storage medium, comprising a stored program, wherein the program, when executed, controls a device in which the computer-readable storage medium is located to perform any of the methods.

[0018] According to still another aspect of the present application, there is provided a processor configured to execute a program, wherein the program, when executed, performs any of the methods.

[0019] According to still another aspect of the present application, there is provided a vehicle, wherein a LiDAR and a camera sensor are installed on the vehicle, and the vehicle comprises a processing unit in communication with the LiDAR and the camera sensor, respectively, and the processing unit is configured to perform any of the methods.

[0020] By applying the technical solution of this application, image data and point cloud data of the area in front of the target vehicle are acquired. A 3D camera model is used to process the image data to obtain a first processing result. A LiDAR model is used to process the point cloud data to obtain a second processing result. The point cloud data is then clustered to obtain a clustering result. Based on the intersection of the first processing result and the clustering result, and / or based on the intersection of the second processing result and the clustering result, it is determined whether the noise in the area below the road sign is a real object. By using a 3D camera model, a LiDAR model, and the results obtained from direct clustering, it is possible to determine whether the noise in the area below the road sign is a real object, further ensuring the accuracy of vehicle path planning. Attached Figure Description

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 A flowchart illustrating a method for identifying noise beneath a road sign according to an embodiment of this application is shown.

[0023] Figure 2 A schematic diagram of an apparatus for identifying noise below a road sign according to an embodiment of this application is shown. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0027] It should be understood that when an element (such as a layer, film, region, or substrate) is described as "on" another element, it can be directly on the other element, or there can be an intermediate element. Also, in the specification and claims, when an element is described as "connected" to another element, it can be "directly connected" to the other element, or "connected" to the other element through a third element.

[0028] As introduced in the background, the existing scheme often identifies the noise points below the road sign as real objects in rainy, foggy and other weather conditions, affecting the accuracy of vehicle path planning. To solve the problem that the existing scheme lacks a method for accurately identifying whether the noise points below the road sign are real objects, embodiments of the present application provide a method, device, computer readable storage medium, processor and vehicle for identifying noise points below a road sign.

[0029] According to an embodiment of the present application, a method for identifying noise points below a road sign is provided.

[0030] Figure 1 is a flowchart of the method for identifying noise points below a road sign according to an embodiment of the present application. As shown in Figure 1 the method includes the following steps:

[0031] Step S101, acquiring image data and point cloud data of a region in front of a target vehicle, the region in front including a road sign;

[0032] Specifically, the image data of the region in front of the vehicle is acquired by using a camera installed on the vehicle;

[0033] Specifically, the point cloud data of the region in front of the vehicle is acquired by using a radar installed on the vehicle;

[0034] It should be noted that the region in front of the target vehicle refers to the region in front of the vehicle, but not only the region including the front of the vehicle;

[0035] Specifically, the road sign can be on the left side, right side or above the vehicle.

[0036] In step S102, the 3D camera model is used to process the image data to obtain a first processing result.

[0037] In step S103, the lidar model is used to process the point cloud data to obtain a second processing result.

[0038] In step S104, the point cloud data is clustered to obtain a clustering result.

[0039] In step S104, the data clustering algorithm is directly used to cluster the point cloud data to obtain the clustering result. Although the point cloud data is also processed, the lidar model is not used but direct clustering is performed. Thus, a rough region can be clustered for the noise region. However, the lidar model generally obtains a label result of a real object. The lidar model is not applicable to the noise region.

[0040] In step S105, whether the noise point under the signboard is a real object is determined according to the intersection of the first processing result and the clustering result, and / or according to the intersection of the second processing result and the clustering result.

[0041] Specifically, the intersection of the first processing result and the clustering result refers to the size of the overlapping part of the first processing result and the clustering result. The intersection of the second processing result and the clustering result refers to the size of the overlapping part of the second processing result and the clustering result.

[0042] In the above scheme, the image data and the point cloud data of the region in front of the target vehicle are obtained. The 3D camera model is used to process the image data to obtain a first processing result. The lidar model is used to process the point cloud data to obtain a second processing result. The point cloud data is clustered to obtain a clustering result. Whether the noise point under the signboard is a real object is determined according to the intersection of the first processing result and the clustering result, and / or according to the intersection of the second processing result and the clustering result. The results obtained by using the 3D camera model, the lidar model, and direct clustering are used to determine whether the noise point under the signboard is a real object, thereby further ensuring the accuracy of vehicle path planning.

[0043] In an optional embodiment, the image data is processed by using a 3D camera model to obtain a first processing result, including: processing the image data by using a 3D camera model to obtain a first bounding box; scaling the first bounding box along the driving direction of the target vehicle according to a scaling ratio to obtain a second bounding box, and the scaling ratio is determined by the distance between the target vehicle and the front target object. That is, the image data is input into the 3D camera model for processing to obtain the first bounding box, which can mark the front target object of the vehicle. To achieve accurate marking of the front target object of the vehicle, the first bounding box is scaled to obtain the second bounding box. Specifically, if the distance between the target vehicle and the front target object is greater than a predetermined distance, the first bounding box is scaled by a predetermined ratio, and if the distance between the target vehicle and the front target object is less than the predetermined distance, the first bounding box is scaled by a predetermined ratio.

[0044] In an optional embodiment, the second processing result is a third bounding box, and the clustering result is an outer bounding box of the shape obtained by clustering. According to the intersection of the first processing result and the clustering result, and / or according to the intersection of the second processing result and the clustering result, it is determined whether the noise point in the area below the signboard is a real object, including: obtaining a first proportion of the volume of the overlapping part of the second bounding box and the outer bounding box to the total volume of the outer bounding box; obtaining a second proportion of the volume of the overlapping part of the third bounding box and the outer bounding box to the total volume of the outer bounding box; in the case that the first proportion is greater than a predetermined proportion, and / or the second proportion is greater than the predetermined proportion, it is determined that the noise point in the area below the signboard is a real object; otherwise, it is determined that the noise point in the area below the signboard is not a real object. That is, according to the size of the volume of the overlapping part of the second bounding box and the outer bounding box, and / or the size of the volume of the overlapping part of the third bounding box and the outer bounding box, it is determined whether the noise point in the area below the signboard is a real object. Specifically, the first bounding box, the second bounding box and the third bounding box are all represented by a prism.

[0045] In an optional embodiment, after it is determined that the noise point in the area below the signboard is not a real object, the method further includes: deleting the noise point in the area below the signboard. That is, if it is only a noise point, the noise point is deleted, which does not represent a real object, thereby ensuring accurate path planning of the vehicle. Especially in the case of a vehicle driving on a highway, it can be ensured that the vehicle will not slow down or brake due to the existence of the noise point, thereby ensuring the driving speed.

[0046] In an optional embodiment, the method further comprises: determining whether the noise points in the area below the signboard are real objects according to the intersection of the first processing result and the clustering result, and / or according to the intersection of the second processing result and the clustering result, when the distance between the target vehicle and the signboard is greater than or equal to a first distance and less than or equal to a second distance, the second distance being greater than the first distance. That is, the scheme of the present application is suitable for processing for a suitable distance, for example, for a target object with a distance of 50m to 120m from the present application.

[0047] In an optional embodiment, the method further comprises: identifying the noise points in the area below the signboard as real objects when the distance between the target vehicle and the signboard is less than the first distance. That is, in the case of a small distance between the target vehicle and the signboard, for example, less than 50m, to ensure the safe driving of the vehicle, the noise points are directly identified as real objects, and the vehicle is controlled to slow down or brake.

[0048] In an optional embodiment, the method further comprises: retaining the noise points in the area below the signboard when the distance between the target vehicle and the signboard is greater than the second distance. That is, for a signboard far away from the vehicle, for example, greater than 120m, the point cloud data monitored by the radar is not accurate enough, and the image captured by the camera may also be inaccurate due to resolution and other reasons, and is no longer suitable for the scheme of the present application.

[0049] In an optional embodiment, the method further comprises: obtaining a first width of the sub-image corresponding to the front target object in the image data; obtaining a second width of the sub-image corresponding to the front target object in the cropped image; and determining that the probability that the image data does not cover the entire front target object is greater than a predetermined probability when the first width is greater than the second width. That is, due to the position of the camera, the shooting angle of the camera, the size of the target object, the distance between the target object and the camera, and other reasons, the probability that the camera captures the full view of the tail of the target object is not high. To ensure the accuracy of determining whether the noise points are real objects, a portion of the object captured by the camera is cropped on both sides (for example, 50 pixels in width), and if the cropped image still includes the full view of the tail of the target object, the scheme of the present application is used again to accurately determine whether the noise points under the signboard are real objects.

[0050] In an optional embodiment, the method further comprises: obtaining the reflectivity of each point in the point cloud data; and retaining the points with reflectivity greater than a predetermined reflectivity. The reflectivity of the point cloud of the real object is relatively high, and retaining the point cloud with high reflectivity can reduce the probability of misjudgment.

[0051] In an optional embodiment, the target vehicle is driving on a highway. The present solution is particularly suitable for vehicles driving on a highway, and in particular, the vehicle can be an autonomous vehicle or an assisted driving vehicle. To ensure the driving speed of the vehicle, for example, to ensure that the truck delivers the goods on time, the present solution is particularly suitable.

[0052] In an optional embodiment, the 3D camera model is a first deep learning model trained using a first convolutional neural network, and the lidar model is a second deep learning model trained using a second convolutional neural network. Using deep learning models further ensures the accuracy of the determination of noise points.

[0053] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0054] The embodiments of the present application also provide a device for identifying noise points under a street sign. It should be noted that the device for identifying noise points under a street sign according to the embodiments of the present application can be used to execute the method for identifying noise points under a street sign provided by the embodiments of the present application. The device for identifying noise points under a street sign provided by the embodiments of the present application is introduced as follows.

[0055] Figure 2 is a schematic diagram of a device for identifying noise points under a street sign according to the embodiments of the present application. As shown in Figure 2 the device comprises:

[0056] A first obtaining unit 10 is configured to obtain image data and point cloud data of a region in front of a target vehicle, wherein the region in front of the target vehicle includes a street sign;

[0057] Specifically, the image data of the region in front of the vehicle is obtained by using a camera installed on the vehicle;

[0058] Specifically, the point cloud data of the region in front of the vehicle is obtained by using a radar installed on the vehicle;

[0059] It should be noted that the region in front of the target vehicle refers to the region in front of the vehicle, but not only the region including the front of the vehicle;

[0060] Specifically, the road sign can be on the left side, the right side or above the vehicle.

[0061] The first processing unit 20 is configured to process the image data by using a 3D camera model to obtain a first processing result.

[0062] The second processing unit 30 is configured to process the point cloud data by using a laser radar model to obtain a second processing result.

[0063] The clustering unit 40 is configured to cluster the point cloud data to obtain a clustering result.

[0064] The point cloud data is directly clustered by using a data clustering algorithm to obtain the clustering result, that is, the point cloud data is also processed, but the laser radar model is not used and the clustering is directly performed, so that a general region can be clustered for the noise point region, but the laser radar model generally obtains a label result of a real object, that is, the laser radar model is not applicable to the noise point region.

[0065] The first determination unit 50 is configured to determine whether the noise point under the road sign is a real object according to an intersection of the first processing result and the clustering result, and / or according to an intersection of the second processing result and the clustering result.

[0066] Specifically, the intersection of the first processing result and the clustering result refers to a size of an overlapping part of the first processing result and the clustering result, and the intersection of the second processing result and the clustering result refers to a size of an overlapping part of the second processing result and the clustering result.

[0067] In the scheme, the first acquisition unit acquires image data and point cloud data of a region in front of a target vehicle, the first processing unit processes the image data by using a 3D camera model to obtain a first processing result, the second processing unit processes the point cloud data by using a laser radar model to obtain a second processing result, the clustering unit clusters the point cloud data to obtain a clustering result, and the first determination unit determines whether a noise point under the road sign is a real object according to an intersection of the first processing result and the clustering result, and / or according to an intersection of the second processing result and the clustering result. The results obtained by using the 3D camera model, the laser radar model and direct clustering are used to determine whether the noise point under the road sign is a real object, and the accuracy of vehicle path planning is further ensured.

[0068] In an alternative embodiment, the first processing unit comprises a processing module and a scaling module, the processing module is configured to process the image data using a 3D camera model to obtain a first bounding box, and the scaling module is configured to scale the first bounding box along the driving direction of the target vehicle according to a scaling ratio to obtain a second bounding box, wherein the scaling ratio is determined by the distance between the target vehicle and the front target object. That is, the image data is input into the 3D camera model for processing to obtain the first bounding box, and the first bounding box can mark the front target object of the vehicle. In order to accurately mark the front target object of the vehicle, the first bounding box is scaled to obtain the second bounding box. Specifically, if the distance between the target vehicle and the front target object is greater than a predetermined distance, the first bounding box is scaled by a predetermined ratio, and if the distance between the target vehicle and the front target object is less than the predetermined distance, the first bounding box is scaled by a predetermined ratio.

[0069] In an alternative embodiment, the second processing result is a third bounding box, and the clustering result is an outer bounding box of the shape obtained by clustering. The first determining unit comprises a first obtaining module, a second obtaining module, a first determining module and a second determining module. The first obtaining module is configured to obtain a first proportion of a volume of an overlapping part of the second bounding box and the outer bounding box to a total volume of the outer bounding box. The second obtaining module is configured to obtain a second proportion of a volume of an overlapping part of the third bounding box and the outer bounding box to the total volume of the outer bounding box. The first determining module is configured to determine that the noise points in the area below the signboard are actually existing objects when the first proportion is greater than a predetermined proportion and / or the second proportion is greater than the predetermined proportion. The second determining module is configured to determine that the noise points in the area below the signboard are not actually existing objects otherwise. That is, whether the noise points in the area below the signboard are actually existing objects is determined according to the size of the volume of the overlapping part of the second bounding box and the outer bounding box and / or the size of the volume of the overlapping part of the third bounding box and the outer bounding box. Specifically, the first bounding box, the second bounding box and the third bounding box are all represented by a prism.

[0070] In an alternative embodiment, the device further comprises a deleting unit configured to delete the noise points in the area below the signboard after determining that the noise points in the area below the signboard are not actually existing objects. That is, if the noise points are only noise points, the noise points are deleted, and thus the accurate path planning of the vehicle is ensured. In particular, when the vehicle is driving on a highway, it can be ensured that the vehicle will not slow down or brake due to the existence of noise points, and thus the driving speed is ensured.

[0071] In an alternative embodiment, the first determining unit comprises a third obtaining module and a third determining module. The third obtaining module is configured to obtain the distance between the target vehicle and the road sign. The third determining module is configured to, when the distance between the target vehicle and the road sign is greater than or equal to a first distance and less than or equal to a second distance, determine whether the noise point in the area below the road sign is a real object according to the intersection of the first processing result and the clustering result, and / or according to the intersection of the second processing result and the clustering result, the second distance being greater than the first distance. That is, the scheme of the present application is suitable for processing for a suitable distance, for example, for a target object with a distance of 50m to 120m, the scheme of the present application is suitable.

[0072] In an alternative embodiment, the device further comprises a determining unit. The determining unit is configured to, when the distance between the target vehicle and the road sign is less than the first distance, determine that the noise point in the area below the road sign is a real object. That is, when the distance between the target vehicle and the road sign is small, for example, less than 50m, in order to ensure the safe driving of the vehicle, the noise point is directly determined to be a real object, and the vehicle is controlled to slow down or brake.

[0073] In an alternative embodiment, the device further comprises a first retaining unit. The first retaining unit is configured to, when the distance between the target vehicle and the road sign is greater than a second distance, retain the noise point in the area below the road sign. That is, for a road sign far away from the vehicle, for example, greater than 120m, the point cloud data monitored by the radar is not accurate enough, and the image captured by the camera may also be inaccurate due to resolution and other reasons, and is no longer suitable for the scheme of the present application.

[0074] In an alternative embodiment, the device further comprises a processing unit, a second acquisition unit, a third acquisition unit and a second determination unit, the processing unit is configured to remove a predetermined width on both sides along the horizontal axis direction of the image data to obtain a cropped image; the second acquisition unit is configured to acquire a first width of a sub-image corresponding to the front target object in the image data; the third acquisition unit is configured to acquire a second width of the sub-image corresponding to the front target object in the cropped image; and the second determination unit is configured to determine that the probability that the image data does not cover the whole of the front target object is greater than a predetermined probability when the first width is greater than the second width. Due to the position of the camera, the shooting angle of the camera, the size of the target object, the distance between the target object and the camera, etc., the probability that the camera can capture the whole of the tail of the target object is not high. In order to ensure the accuracy of determining whether the noise point is a real object, a part of the object captured by the camera is cropped on both sides (for example, 50 pixels in width are cropped on each side), and if the cropped image still includes the whole of the tail of the target object, the scheme of the present application can be used to accurately determine whether the noise point under the signboard is a real object.

[0075] In an alternative embodiment, the device further comprises a fourth acquisition unit and a second reservation unit, the fourth acquisition unit is configured to acquire the reflectivity of each point in the point cloud data; and the second reservation unit is configured to reserve points with a reflectivity greater than a predetermined reflectivity. The reflectivity of the point cloud of a real object is relatively high, and reserving the point cloud with high reflectivity can reduce the probability of misjudgment.

[0076] The device for identifying noise points under a signboard comprises a processor and a memory, and the first acquisition unit, the first processing unit, the second processing unit, the clustering unit and the first determination unit are all stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0077] The processor comprises a core, and the core retrieves the corresponding program units from the memory. The core can be one or more, and the core parameters are adjusted to determine whether the noise points under the signboard are real objects.

[0078] The memory can include non-persistent memory in a computer readable medium, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.

[0079] The embodiment of the present application provides a computer readable storage medium, which comprises a stored program, wherein when the program runs, the device where the computer readable storage medium is located is controlled to execute the method for identifying noise points under a signboard.

[0080] The embodiment of the present application provides a processor used for running a program, wherein the processor executes the method for identifying the noise points under the road sign when the program is running.

[0081] The embodiment of the present application provides a vehicle, wherein the vehicle is provided with a laser radar and a camera sensor, and the vehicle comprises a processing unit, the processing unit is in communication with the laser radar and the camera sensor respectively, and the processing unit is used for executing any one of the methods.

[0082] The embodiment of the present application provides a device, the device comprising a processor, a memory and a program stored in the memory and executable on the processor, and the processor executes the program to implement at least the following steps:

[0083] Step S101: acquiring image data and point cloud data of a front area of a target vehicle, wherein the front area comprises a road sign;

[0084] Step S102: processing the image data by using a 3D camera model to obtain a first processing result;

[0085] Step S103: processing the point cloud data by using a laser radar model to obtain a second processing result;

[0086] Step S104: clustering the point cloud data to obtain a clustering result;

[0087] Step S105: determining whether the noise points in the area under the road sign are real objects according to an intersection of the first processing result and the clustering result, and / or according to an intersection of the second processing result and the clustering result.

[0088] The device herein can be a server, a PC, a PAD, a mobile phone or the like.

[0089] The present application also provides a computer program product, when executed on a data processing device, is suitable for executing a program initialized with at least the following method steps:

[0090] Step S101: acquiring image data and point cloud data of a front area of a target vehicle, wherein the front area comprises a road sign;

[0091] Step S102: processing the image data by using a 3D camera model to obtain a first processing result;

[0092] Step S103: processing the point cloud data by using a laser radar model to obtain a second processing result;

[0093] Step S104: clustering the point cloud data to obtain a clustering result;

[0094] Step S105, according to the intersection of the first processing result and the clustering result, and / or according to the intersection of the second processing result and the clustering result, determine whether the noise points of the area below the signboard are real objects.

[0095] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems, or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0096] The present application is described in reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more flows and / or blocks.

[0097] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more flows and / or blocks.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide functions for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more flows and / or blocks.

[0099] In a typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memories.

[0100] Memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or non-volatile memory of other types in the nature of static or dynamic RAM. Memory is an example of computer readable storage media. A "computer-readable storage medium" can be any available medium or

[0101] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media such as modulated data signals and carrier waves.

[0102] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0103] From the above description, it can be seen that the above-mentioned embodiments of the present application achieve the following technical effects:

[0104] 1)、The method for identifying the noise points below the signboard of the application, by obtaining the image data and point cloud data of the region in front of the target vehicle, using a 3D camera model to process the image data to obtain a first processing result, using a laser radar model to process the point cloud data to obtain a second processing result, clustering the point cloud data to obtain a clustering result, and determining whether the noise points in the region below the signboard are real objects according to the intersection of the first processing result and the clustering result, and / or according to the intersection of the second processing result and the clustering result. The results obtained by using the 3D camera model, the laser radar model and direct clustering realize the determination of whether the noise points in the region below the signboard are real objects, further ensuring the accuracy of vehicle path planning.

[0105] 2)、The device for identifying the noise points below the signboard of the application, the first acquisition unit obtains the image data and point cloud data of the region in front of the target vehicle, the first processing unit uses a 3D camera model to process the image data to obtain a first processing result, the second processing unit uses a laser radar model to process the point cloud data to obtain a second processing result, the clustering unit clusters the point cloud data to obtain a clustering result, and the first determination unit determines whether the noise points in the region below the signboard are real objects according to the intersection of the first processing result and the clustering result, and / or according to the intersection of the second processing result and the clustering result. The results obtained by using the 3D camera model, the laser radar model and direct clustering realize the determination of whether the noise points in the region below the signboard are real objects, further ensuring the accuracy of vehicle path planning.

[0106] The above only describes the preferred embodiments of the application and is not intended to limit the application. Those skilled in the art can make various modifications and changes to the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A method for identifying noise points below road signs, characterized in that, include: Acquire image data and point cloud data of the area in front of the target vehicle, wherein the area in front includes road signs; The image data is processed using a 3D camera model to obtain a first processing result; The point cloud data is processed using a lidar model to obtain a second processing result; The point cloud data is clustered to obtain the clustering results; Based on the intersection of the first processing result and the clustering result, and / or based on the intersection of the second processing result and the clustering result, determine whether the noise in the area below the road sign is a real object; Based on the first processing result, the second processing result, and the clustering result, determine whether the noise in the area below the road sign is a real object, including: obtaining the distance between the target vehicle and the road sign; If the distance between the target vehicle and the road sign is greater than or equal to a first distance and less than or equal to a second distance, the noise in the area below the road sign is determined to be a real object based on the intersection of the first processing result and the clustering result, and / or based on the intersection of the second processing result and the clustering result, wherein the second distance is greater than the first distance; if the distance between the target vehicle and the road sign is less than the first distance, the noise in the area below the road sign is identified as a real object; if the distance between the target vehicle and the road sign is greater than the second distance, the noise in the area below the road sign is retained.

2. The method according to claim 1, characterized in that, The image data is processed using a 3D camera model to obtain a first processing result, including: The image data is processed using a 3D camera model to obtain the first bounding box; The first annotation box is scaled along the driving direction of the target vehicle according to a scaling ratio to obtain the second annotation box. The scaling ratio is determined by the distance between the target vehicle and the target object in front.

3. The method according to claim 2, characterized in that, The second processing result is the third annotation box, and the clustering result is the bounding box of the shape obtained by clustering. Based on the intersection of the first processing result and the clustering result, and / or based on the intersection of the second processing result and the clustering result, it is determined whether the noise in the area below the road sign is a real object, including: Obtain the first percentage of the volume of the overlapping portion of the second annotation box and the outer bounding box to the total volume of the outer bounding box; Obtain the second percentage of the volume of the overlapping portion of the third annotation box and the outer bounding box relative to the total volume of the outer bounding box; If the first proportion is greater than the predetermined proportion, and / or the second proportion is greater than the predetermined proportion, it is determined that the noise in the area below the road sign is a real object. Otherwise, it is determined that the noise in the area below the road sign is not a real object.

4. The method according to claim 3, characterized in that, After determining that the noise in the area below the road sign is not a real object, the method further includes: Remove noise from the area below the road sign.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Along the horizontal axis of the image data, a predetermined width is removed from both sides to obtain the cropped image; Obtain the first width of the sub-image corresponding to the target object in front in the image data; Obtain the second width of the sub-image corresponding to the target object in the cropped image; When the first width is greater than the second width, the probability that the image data does not cover the entirety of the target object in front is greater than a predetermined probability.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Obtain the reflectance of each point in the point cloud data; Points with reflectivity greater than the predetermined reflectivity are retained.

7. The method according to any one of claims 1 to 6, characterized in that, The target vehicle was traveling on a highway.

8. The method according to any one of claims 1 to 6, characterized in that, The 3D camera model is a first deep learning model, which is trained using a first convolutional neural network, and the lidar model is a second deep learning model, which is trained using a second convolutional neural network.

9. A device for identifying noise points below road signs, characterized in that, include: The first acquisition unit is used to acquire image data and point cloud data of the area in front of the target vehicle, wherein the area in front includes road signs; The first processing unit is used to process the image data using a 3D camera model to obtain a first processing result; The second processing unit is used to process the point cloud data using a lidar model to obtain a second processing result; Clustering unit, used to cluster the point cloud data to obtain clustering results; The first determining unit is configured to determine whether the noise in the area below the road sign is a real object based on the intersection of the first processing result and the clustering result, and / or based on the intersection of the second processing result and the clustering result. The first determining unit includes a third acquisition module and a third determining module. The third acquisition module is used to acquire the distance between the target vehicle and the road sign. The third determining module is used to determine whether the noise in the area below the road sign is a real object, based on the intersection of the first processing result and the clustering result, and / or based on the intersection of the second processing result and the clustering result, when the distance between the target vehicle and the road sign is greater than or equal to a first distance and less than or equal to a second distance, and the second distance is greater than the first distance. The device further includes a recognition unit, which is used to recognize noise in the area below the road sign as a real object when the distance between the target vehicle and the road sign is less than the first distance; the device further includes a first retention unit, which is used to retain noise in the area below the road sign when the distance between the target vehicle and the road sign is greater than the second distance.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 8.

11. A processor, characterized in that, The processor is used to run a program, wherein the program, when running, performs the method according to any one of claims 1 to 8.

12. A vehicle, characterized in that, The vehicle is equipped with a lidar and a camera sensor, and the vehicle includes a processing unit that communicates with the lidar and the camera sensor respectively. The processing unit is used to perform the method according to any one of claims 1 to 8.

Images