A method, device, computer device, and storage medium for identifying missed inspection of an object

Abstract

This application relates to a method, device, computer device, and storage medium for identifying missed detection of objects. The method includes: obtaining first object detection data, where the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected; obtaining second object detection data, where the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle; determining whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data. By using this method, the phenomenon of missed detection of objects can be identified, thereby optimizing the perception part of the autonomous driving vehicle and improving the perception accuracy of the autonomous driving vehicle for the environment.

Description

Technical Field

[0001] The present application relates to the technical field of autonomous driving, and particularly to a method, device, computer device, and storage medium for identifying missed detection of objects. Background Art

[0002] A lidar is a device that uses lasers for detection and ranging. Currently, due to its reliability, the lidar is the most important sensor in an unmanned driving system. In actual application scenarios, the lidar is not perfect. For example, the lidar has problems such as overly sparse point clouds or even partial loss of point clouds. Another example is that it is difficult to distinguish the surface of irregular objects based on lidar point clouds, and it is difficult to use the lidar in situations such as heavy rain.

[0003] Therefore, currently, an unmanned driving system often fuses a lidar and a camera. The high resolution of the camera is used to classify targets, and the reliability of the lidar is used to detect and range the environment. The advantages of both are combined to complete the environmental perception task.

[0004] Since the detection distance of the camera is limited by the light intensity and is not applicable to long-distance detection, when transmitting the two-dimensional image data collected by the camera to an object detection model for object detection, there is a phenomenon of missed detection of objects. In order to improve the accuracy of environmental perception, it is necessary to detect or identify this missed detection phenomenon for targeted optimization. Summary of the Invention

[0005] Based on this, a method, device, computer device, and storage medium for identifying missed detection of objects are provided to identify the missed detection phenomenon of objects by an image object detection model.

[0006] In a first aspect, a method for identifying missed detection of objects is provided. The method includes:

[0007] Obtain first object detection data, where the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected;

[0008] Obtain second object detection data, where the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle;

[0009] Determine whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data.

[0010] In a first aspect, in combination with the first implementable manner of the first aspect, the step of determining whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data includes:

[0011] Project the first object detection data onto the second object detection data to obtain the projection data to be detected;

[0012] Obtain the first object overlap degree according to the projection data to be detected and the second object detection data;

[0013] Compare the first object overlap degree with a preset overlap degree threshold to determine whether the first object overlap degree is greater than the overlap degree threshold;

[0014] If not, determine that there is a missed detection in the second object detection data;

[0015] If so, determine that there is no missed detection in the second object detection data.

[0016] In a first aspect, in combination with the second implementable manner of the first aspect, the step of projecting the first object detection data onto the second object detection data to obtain the projection data to be detected includes:

[0017] Obtain each first object to be detected in the first object detection data, where the manifestation form of the first object to be detected includes a three-dimensional polygon;

[0018] Sample the three-dimensional polygon to obtain at least one key point, where the key point includes the vertex of the three-dimensional polygon frame;

[0019] Project each key point onto the camera coordinate system through the external camera parameters to obtain corresponding first projection points;

[0020] Project each first projection point onto the two-dimensional imaging plane of the camera through the internal camera parameters to obtain corresponding second projection points;

[0021] Surround each second projection point to obtain a projection polygon;

[0022] Fuse the projection polygons of each three-dimensional polygon to obtain the projection data to be detected.

[0023] In a first aspect, in combination with the third implementable manner of the first aspect, it further includes:

[0024] Identify the first laser point cloud data to determine whether the first laser point cloud data includes a toll station;

[0025] If not, continue to execute the recognition step of whether there is any undetected second object detection data;

[0026] If so, obtain the distance between the toll station and the current vehicle, and determine whether the distance is less than a preset distance threshold;

[0027] If so, stop executing the recognition step of whether there is any undetected second object detection data;

[0028] If not, continue to execute the recognition step of whether there is any undetected second object detection data.

[0029] In the first aspect, in combination with the fourth feasible implementation manner of the first aspect, it further includes:

[0030] Obtain each of the first objects to be detected in the first object detection data including at least one three-dimensional polygon, wherein the representation form of the first object to be detected includes a three-dimensional polygon;

[0031] Obtain the first extreme point of each of the three-dimensional polygons in the first direction and the second extreme point in the second direction, wherein the first direction is used to indicate the direction perpendicular to the laser emission direction, and the second direction is used to indicate the direction parallel to the ground;

[0032] Expand the first extreme point according to a preset expansion distance to obtain a first expansion point, and expand the second extreme point according to the expansion distance to obtain a second expansion point;

[0033] Obtain the first reflection distance of the laser at the first expansion point and the second reflection distance of the laser at the second expansion point;

[0034] Compare the first reflection distance and the second reflection distance with the length of the three-dimensional polygon respectively, and determine whether both the first reflection distance and the second reflection distance are greater than the length of the three-dimensional polygon;

[0035] If so, determine that the three-dimensional polygon is not blocked, and continue to execute the recognition step of whether there is any undetected second object detection data;

[0036] If not, determine that the three-dimensional polygon is blocked, and stop executing the recognition step of whether there is any undetected second object detection data.

[0037] In the first aspect, in combination with the fifth feasible implementation manner of the first aspect, after the step of projecting the first object detection data onto the second object detection data to obtain the data to be detected for projection, it includes:

[0038] Obtain first target object detection data, where the first target object detection data is obtained by detecting second lidar point cloud data including at least one target object, and the distance from the target object to the current vehicle is less than the distance from the first object to be detected to the current vehicle;

[0039] Obtain second target object detection data, where the second target object detection data is obtained by detecting second camera image data including at least one of the target objects;

[0040] Project the first target object detection data onto the second target object detection data to obtain target projection data;

[0041] Obtain a second object overlap degree based on the projection data to be detected and the target projection data;

[0042] Compare the second object overlap degree with the overlap degree threshold to determine whether the second object overlap degree is greater than the overlap degree threshold;

[0043] If so, stop executing the recognition step of whether there is a missed detection in the second object detection data;

[0044] If not, continue to execute the recognition step of whether there is a missed detection in the second object detection data.

[0045] In a first aspect, in combination with the sixth implementable manner of the first aspect, after the step of projecting the first object detection data onto the second object detection data to obtain projection data to be detected, it includes:

[0046] Obtain a first pixel and a second pixel of the projection data to be detected, where the first pixel is used to indicate the pixel of the projection polygon in the length direction in the projection data to be detected, and the second pixel is used to indicate the pixel of the projection polygon in the width direction in the projection data to be detected;

[0047] Compare the first pixel and the second pixel with a preset pixel threshold respectively to determine whether both the first pixel and the second pixel are greater than the pixel threshold;

[0048] If so, continue to execute the recognition step of whether there is a missed detection in the second object detection data;

[0049] If not, stop executing the recognition step of whether there is a missed detection in the second object detection data.

[0050] In a first aspect, in combination with the seventh implementable manner of the first aspect, the step of obtaining the first object detection data includes:

[0051] Obtain the detection range of the current vehicle and the first lidar point cloud data including at least one first object to be detected within the detection range;

[0052] Based on the map information, filter the first lidar point cloud data to obtain preprocessed point cloud data including lanes;

[0053] Cluster and enclose the preprocessed point cloud data to obtain three-dimensional polygons corresponding to each of the first objects to be detected and first object detection data including each of the three-dimensional polygons.

[0054] In a second aspect, there is provided an identification device for object undetected, the device comprising:

[0055] A first acquisition unit configured to acquire first object detection data, where the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected;

[0056] A second acquisition unit configured to acquire second object detection data, where the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle;

[0057] A processing unit configured to determine whether there is an undetected object in the second object detection data according to the overlap degree between the first object detection data and the second object detection data.

[0058] In a third aspect, there is provided a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the steps of the object undetected identification method according to any one of the first aspect or the implementable manners combined with the first aspect are implemented.

[0059] In a fourth aspect, there is provided a computer-readable storage medium, on which a computer program is stored, where when the computer program is executed by a processor, the steps of the object undetected identification method according to any one of the first aspect or the implementable manners combined with the first aspect are implemented.

[0060] The method, apparatus, computer device, and storage medium for identifying missed detection of the above-mentioned object obtain first object detection data and second object detection data. Among them, the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected, and the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle. Then, the first object detection data and the second object detection data are compared, and according to the overlap degree of the first object detection data and the second object detection data, it is determined whether there is a missed detection in the second object detection data. By using the above method to identify the missed detection phenomenon of an object by an image object detection model, the perception part of an autonomous driving vehicle is optimized, and the perception accuracy of the autonomous driving vehicle for the environment is improved. Description of the Drawings

[0061] Figure 1 It is a schematic flowchart of a method for identifying missed detection of an object in an embodiment;

[0062] Figure 2 It is a structural block diagram of a device for identifying missed detection of an object in an embodiment;

[0063] Figure 3 It is an internal structure diagram of a computer device in an embodiment. Detailed Description of the Embodiment

[0064] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] It should be noted that the diagrams provided in this embodiment only illustrate the basic concept of the present application schematically. Therefore, only the components related to the present application are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0066] The structures, ratios, sizes, etc. shown in the drawings of this specification are only used to cooperate with the content disclosed in the specification for those skilled in the art to understand and read, and are not used to limit the limited conditions under which the present application can be implemented. Therefore, they do not have technical substance. Any modification of the structure, change of the proportional relationship, or adjustment of the size should still fall within the scope that can be covered by the technical content disclosed in the present application without affecting the effects that the present application can produce and the objectives that can be achieved.

[0067] The orientation or positional relationship indicated by terms such as "upper", "lower", "left", "right", "middle", "longitudinal", "lateral", "horizontal", "inner", "outer", "radial", "circumferential", etc., cited in this specification is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of simplified description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation on this application. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0068] Currently, the laser point cloud data collected by lidar is usually detected to obtain lidar point cloud detection data, and the image data collected by the camera is detected to obtain camera image detection data; then the lidar point cloud detection data and the camera image detection data are fused to complete environmental perception and ranging. However, when detecting the image data collected by the camera, due to the limited detection ability of the camera, there is a phenomenon of missed detection of objects when transmitting the two-dimensional image data collected by the camera to the object detection model for object detection, which affects the accuracy of environmental perception.

[0069] Therefore, this application proposes a method, device, computer device and storage medium for identifying missed detection of objects. By obtaining first object detection data, where the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected; obtaining second object detection data, where the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle; determining whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data. Since lidar has reliability in environmental detection and ranging, and the detection distance of the camera is limited by light intensity, the detection accuracy for distant objects needs to be improved. Therefore, through the above method for identifying missed detection of objects, the first object detection data obtained by detecting the lidar point cloud data is used as a reference to determine whether there is a missed detection in the second object detection data obtained by detecting the camera image data, so that engineers can optimize the image object detection model according to the existing missed detection phenomenon and improve the accuracy of environmental perception of autonomous vehicles.

[0070] In one embodiment, as Figure 1 shown, a method for identifying missed detection of objects is provided, including the following steps:

[0071] S101: Obtain first object detection data, where the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected.

[0072] The pulsed laser is emitted by the lidar of the current vehicle. The pulsed laser is emitted to each of the first objects to be detected, causing scattering, and a part of the laser wave will be reflected back to the lidar. Then, according to the principle of laser ranging, the distances from the lidar to each of the first objects to be detected are calculated. The pulsed laser continuously scans each of the first objects to be detected to obtain the detection point data of each of the first objects to be detected. Based on the detection point data of each of the first objects to be detected, imaging processing is performed to obtain the first laser point cloud data. Exemplarily, the Flood Fill algorithm can be used to detect the first laser point cloud data, thereby obtaining the first object detection data.

[0073] S102: Obtain second object detection data, where the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle.

[0074] The first camera image data is collected by the camera for each of the second objects to be detected. Exemplarily, an image object detection model based on deep learning, such as the Yolo model, can be used to detect the first camera image data, thereby obtaining the second object detection data.

[0075] S103: Determine whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data.

[0076] In the above method for identifying missed detection of objects, by obtaining first object detection data and second object detection data, where the first object detection data is obtained by detecting first laser point cloud data including at least one first object to be detected, the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle; then comparing the first object detection data and the second object detection data, and determining whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data.

[0077] Since lidar has reliability in environmental detection and ranging, while the detection distance of a camera is limited by light intensity, the detection accuracy for distant objects needs to be improved. Therefore, through the above method for identifying missed detections of objects, the first object detection data obtained by detecting based on the first lidar point cloud data is used as a reference to determine whether there is a missed detection in the second object detection data obtained by detecting based on the first camera image data, so that engineers can optimize the image object detection model according to the existing missed detections and improve the accuracy of environmental perception of autonomous vehicles.

[0078] As a specific implementation manner of the above embodiment, the step of determining whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data includes:

[0079] Project the first object detection data onto the second object detection data to obtain the projection data to be detected;

[0080] Based on the projection data to be detected and the second object detection data, obtain the first object overlap degree;

[0081] Compare the first object overlap degree with a preset overlap degree threshold to determine whether the first object overlap degree is greater than the overlap degree threshold;

[0082] If not, determine that there is a missed detection in the second object detection data;

[0083] If so, determine that there is no missed detection in the second object detection data.

[0084] It should be noted that each of the first objects to be detected for obtaining the projection data to be detected, where the manifestation form of the first object to be detected includes a projection polygon; and each of the second objects to be detected for the second object detection data is obtained, where the manifestation form of the second object to be detected includes a two-dimensional polygon; based on each projection polygon of the projection data to be detected and each two-dimensional polygon of the second object detection data, the intersection area and the union area can be obtained; by calculating the quotient of the intersection area and the union area, the first object overlap degree can be obtained. Exemplarily, the overlap degree threshold can be set to 0.3. If the first object overlap degree is greater than 0.3, it is confirmed that the second object to be detected is detected, that is, there is no missed detection; otherwise, it is confirmed that the second object to be detected is not detected, that is, there is a missed detection.

[0085] As a specific implementation manner of the above embodiment, the step of projecting the first object detection data onto the second object detection data to obtain the projection data to be detected includes:

[0086] Obtain each of the first objects to be detected for the first object detection data, where the manifestation form of the first object to be detected includes a three-dimensional polygon;

[0087] Sample the three-dimensional polygon to obtain at least one key point, where the key point includes the vertex of the three-dimensional polygon frame;

[0088] Project each of the key points into the camera coordinate system through the external camera parameters to obtain corresponding first projection points;

[0089] Project each of the first projection points into the two-dimensional imaging plane of the camera through the internal camera parameters to obtain corresponding second projection points;

[0090] Surround each of the second projection points to obtain a projected polygon;

[0091] Fuse the projected polygons of each of the three-dimensional polygons to obtain the projected data to be detected.

[0092] Preferably, in one embodiment, the method further includes:

[0093] Identify the first lidar point cloud data to determine whether the first lidar point cloud data includes a toll station;

[0094] If not, continue to execute the identification step of whether there is an undetected object in the second object detection data;

[0095] If so, obtain the distance between the toll station and the current vehicle, and determine whether the distance is less than a preset distance threshold;

[0096] If so, stop executing the identification step of whether there is an undetected object in the second object detection data;

[0097] If not, continue to execute the identification step of whether there is an undetected object in the second object detection data.

[0098] It should be noted that since the toll booth of the toll station is similar to that of an ordinary truck in the lidar point cloud data but does not fall within the scope of camera detection, it is necessary to determine whether the first lidar point cloud data includes a toll station. If so, obtain the distance between the toll station and the current vehicle, and compare the distance with the distance threshold. If the distance is less than the distance threshold, do not execute the step of the identification method for whether there is an undetected object, thereby improving the identification efficiency and accuracy.

[0099] Preferably, in one embodiment, the method further includes:

[0100] Obtaining the first object detection data includes each of the first objects to be detected that includes at least one three-dimensional polygon, wherein the representation form of the first object to be detected includes a three-dimensional polygon;

[0101] Obtaining a first extreme point of each of the three-dimensional polygons in a first direction and a second extreme point in a second direction, wherein the first direction is used to indicate a direction perpendicular to the laser emission direction, and the second direction is used to indicate a direction parallel to the ground;

[0102] Expanding the first extreme point according to a preset expansion distance to obtain a first expansion point, and expanding the second extreme point according to the expansion distance to obtain a second expansion point;

[0103] Obtaining a first reflection distance of the laser at the first expansion point and a second reflection distance of the laser at the second expansion point;

[0104] Comparing the first reflection distance and the second reflection distance with the length of the three-dimensional polygon respectively to determine whether both the first reflection distance and the second reflection distance are greater than the length of the three-dimensional polygon;

[0105] If so, determining that the three-dimensional polygon is not blocked, and continuing to execute the recognition step of whether there is a missed detection in the second object detection data;

[0106] If not, determining that the three-dimensional polygon is blocked, and stopping the execution of the recognition step of whether there is a missed detection in the second object detection data.

[0107] It should be noted that when an object is blocked, especially in the case of severe occlusion, it is prone to be missed detected. However, due to the presence of the occluder, the object generally does not come into direct contact with the current vehicle. Therefore, in this case, even if there is a missed detection phenomenon, the impact on the decision-making of the autonomous vehicle based on the perception information is very small, or even negligible. Therefore, in the case where the object is blocked, the steps of the recognition method for object missed detection can be not executed, so as to filter out the noise data and improve the recognition efficiency. Through the above steps, it can be determined whether there is an occlusion in the three-dimensional polygon corresponding to the first object to be detected. If so, it is not necessary to use the first object to be detected as a reference to determine whether the second object detection data includes a second object to be detected that is consistent with the first object to be detected; otherwise, it is necessary to use the first object to be detected as a reference to determine whether the second object detection data includes a second object to be detected that is consistent with the first object to be detected.

[0108] Preferably, in one embodiment, after the step of projecting the first object detection data onto the second object detection data to obtain the data to be detected for projection, it includes:

[0109] Obtain first target object detection data, where the first target object detection data is obtained by detecting second lidar point cloud data including at least one target object, and the distance from the target object to the current vehicle is less than the distance from the first object to be detected to the current vehicle;

[0110] Obtain second target object detection data, where the second target object detection data is obtained by detecting second camera image data including at least one of the target objects;

[0111] Project the first target object detection data onto the second target object detection data to obtain target projection data;

[0112] Obtain a second object overlap degree based on the projection data to be detected and the target projection data;

[0113] Compare the second object overlap degree with the overlap degree threshold to determine whether the second object overlap degree is greater than the overlap degree threshold;

[0114] If so, stop executing the recognition step of whether there is a missed detection in the second object detection data;

[0115] If not, continue to execute the recognition step of whether there is a missed detection in the second object detection data.

[0116] It should be noted that compared with the first object to be detected, the target object is closer to the current vehicle. By obtaining the second lidar point cloud data including at least one of the target objects, the second lidar point cloud data can be detected by the Flood Fill algorithm to obtain the first target object detection data. Then, by obtaining the second camera image data including at least one of the target objects, the second camera image data can be detected by the Yolo object detection model to obtain the second target object detection data. Then project the first target object detection data onto the second target object detection data to obtain target projection data; based on the projection data to be detected and the target projection data of the target object closer to the current vehicle, obtain the second object overlap degree, and then compare the second object overlap degree with the overlap degree threshold. If the second object overlap degree is less than or equal to the overlap degree threshold, it is determined that the first object to be detected corresponding to the projection data to be detected is not blocked by the target object, and it is necessary to use the first object to be detected as a reference to determine whether the second object detection data includes a second object to be detected that is the same as the first object to be detected; otherwise, it is not necessary to use the first object to be detected as a reference to determine whether the second object detection data includes a second object to be detected that is the same as the first object to be detected, which improves the recognition efficiency to a certain extent.

[0117] Preferably, in one embodiment, after the step of projecting the first object detection data onto the second object detection data to obtain the detection-projection data to be detected, the following steps are included:

[0118] Obtain a first pixel and a second pixel of the detection-projection data to be detected, where the first pixel is used to indicate the pixels of the projected polygon in the length direction in the detection-projection data to be detected, and the second pixel is used to indicate the pixels of the projected polygon in the width direction in the detection-projection data to be detected;

[0119] Compare the first pixel and the second pixel with a preset pixel threshold respectively to determine whether both the first pixel and the second pixel are greater than the pixel threshold;

[0120] If so, continue to execute the recognition step of whether there is an undetected object in the second object detection data;

[0121] If not, stop executing the recognition step of whether there is an undetected object in the second object detection data.

[0122] It should be noted that for the detection-projection data to be detected, it is necessary to obtain its first pixel and second pixel, where the first pixel is used to indicate the pixels of the projected polygon in the length direction in the detection-projection data to be detected, and the second pixel is used to indicate the pixels of the projected polygon in the width direction in the detection-projection data to be detected; then compare the first pixel and the second pixel with the pixel threshold respectively. Only when both the first pixel and the second pixel are greater than the pixel threshold, will the steps of the recognition method for whether there is an undetected object be continued; otherwise, it is considered that the first object to be detected is small. Even if the first object to be detected is used as a reference to determine that there is an undetected object in the second object detection data, it has little impact on the decision-making of the autonomous vehicle based on the perception information and can even be ignored. Therefore, in this case, there is no need to continue to execute the steps of the recognition method for whether there is an undetected object, which further improves the recognition efficiency.

[0123] As a specific implementation manner of the above embodiment, the step of obtaining the first object detection data includes:

[0124] Obtain the detection range of the current vehicle and the first laser point cloud data including at least one first object to be detected within the detection range;

[0125] Based on the map information, filter the first laser point cloud data to obtain the preprocessed point cloud data including the lane;

[0126] Cluster and enclose the preprocessed point cloud data to obtain a three-dimensional polygon corresponding to each of the first objects to be detected, and first object detection data including each of the three-dimensional polygons.

[0127] It should be noted that when the current vehicle perceives environmental information, there is a detection range. By way of example, the lidar of the current vehicle can collect environmental information within a detection range formed by 50 meters in the forward direction, 20 meters in the backward direction, and 10 meters in the left and right directions, so as to obtain first lidar point cloud data including at least one first object to be detected; then, through filtering, clustering, and enclosing by the above method, redundant data is removed to obtain the first object detection data.

[0128] It should be understood that although Figure 1 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figure 1 at least a part of the steps in

[0129] In one embodiment, as Figure 2 shown, an object missed detection recognition device is provided, including:

[0130] A first acquisition unit, which is used to acquire first object detection data, where the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected;

[0131] A second acquisition unit, which is used to acquire second object detection data, where the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle;

[0132] A processing unit, which is used to determine whether there is a missed detection in the second object detection data according to the overlap degree of the first object detection data and the second object detection data.

[0133] It should be noted that the first acquisition unit includes a lidar for acquiring first lidar point cloud data and a lidar point cloud object detection sub-unit for detecting the first lidar point cloud data. Among them, the lidar point cloud object detection sub-unit can embed the Flood Fill algorithm; the second acquisition unit includes a camera for acquiring first camera image data and an image object detection sub-unit for detecting the first camera image data. Among them, the image object sub-unit can embed an image object detection model, such as the Yolo model; the processing unit respectively receives the first object detection data of the first acquisition unit and the second object detection data of the second acquisition unit, and outputs the result of whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data.

[0134] Specifically, the step of the processing unit determining whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data includes:

[0135] Project the first object detection data onto the second object detection data to obtain the to-be-detected projection data;

[0136] According to the to-be-detected projection data and the second object detection data, obtain the first object overlap degree;

[0137] Compare the first object overlap degree with a preset overlap degree threshold to determine whether the first object overlap degree is greater than the overlap degree threshold;

[0138] If not, determine that there is a missed detection in the second object detection data;

[0139] If so, determine that there is no missed detection in the second object detection data.

[0140] Specifically, the step of the processing unit projecting the first object detection data onto the second object detection data to obtain the to-be-detected projection data includes:

[0141] Obtain each first to-be-detected object of the first object detection data, where the manifestation form of the first to-be-detected object includes a three-dimensional polygon;

[0142] Sample the three-dimensional polygon to obtain at least one key point, where the key point includes the vertex of the three-dimensional polygon frame;

[0143] Project each key point onto the camera coordinate system through the external camera parameters to obtain the corresponding first projection point;

[0144] Project each of the first projection points onto the two-dimensional imaging plane of the camera through the camera internal parameters to obtain corresponding second projection points;

[0145] Surround each of the second projection points to obtain a projection polygon;

[0146] Fuse the projection polygons of each of the three-dimensional polygons to obtain the projection data to be detected.

[0147] Preferably, the first acquisition unit is further configured to:

[0148] Identify the first lidar point cloud data to determine whether the first lidar point cloud data includes a toll station;

[0149] If not, continue to perform the identification step of whether there is an undetected object in the second object detection data;

[0150] If so, obtain the distance between the toll station and the current vehicle, and determine whether the distance is less than a preset distance threshold;

[0151] If so, stop performing the identification step of whether there is an undetected object in the second object detection data;

[0152] If not, continue to perform the identification step of whether there is an undetected object in the second object detection data.

[0153] Preferably, the processing unit is further configured to:

[0154] Obtain each of the first objects to be detected included in the first object detection data, where the first objects to be detected are in the form of three-dimensional polygons;

[0155] Obtain a first extreme point of each of the three-dimensional polygons in a first direction and a second extreme point in a second direction, where the first direction is used to indicate a direction perpendicular to the laser emission direction, and the second direction is used to indicate a direction parallel to the ground;

[0156] Expand the first extreme point according to a preset expansion distance to obtain a first expansion point, and expand the second extreme point according to the expansion distance to obtain a second expansion point;

[0157] Obtain a first reflection distance of the laser at the first expansion point and a second reflection distance of the laser at the second expansion point;

[0158] Compare the first reflection distance and the second reflection distance with the length of the three-dimensional polygon respectively to determine whether both the first reflection distance and the second reflection distance are greater than the length of the three-dimensional polygon;

[0159] If so, determine that the three-dimensional polygon is not occluded, and continue to execute the identification step of whether there is any undetected object in the second object detection data;

[0160] If not, determine that the three-dimensional polygon is occluded, and stop executing the identification step of whether there is any undetected object in the second object detection data.

[0161] Preferably, after the step that the processing unit projects the first object detection data onto the second object detection data to obtain the to-be-detected projection data, the following steps are included:

[0162] Obtain first target object detection data, where the first target object detection data is obtained by detecting second laser point cloud data including at least one target object, and the distance from the target object to the current vehicle is less than the distance from the first object to be detected to the current vehicle;

[0163] Obtain second target object detection data, where the second target object detection data is obtained by detecting second camera image data including at least one of the target objects;

[0164] Project the first target object detection data onto the second target object detection data to obtain target projection data;

[0165] Obtain a second object overlap degree according to the to-be-detected projection data and the target projection data;

[0166] Compare the second object overlap degree with the overlap degree threshold, and determine whether the second object overlap degree is greater than the overlap degree threshold;

[0167] If so, stop executing the identification step of whether there is any undetected object in the second object detection data;

[0168] If not, continue to execute the identification step of whether there is any undetected object in the second object detection data.

[0169] Preferably, after the step that the processing unit projects the first object detection data onto the second object detection data to obtain the to-be-detected projection data, the following steps are included:

[0170] Obtain a first pixel and a second pixel of the to-be-detected projection data, where the first pixel is used to indicate the pixel of the projection polygon in the length direction in the to-be-detected projection data, and the second pixel is used to indicate the pixel of the projection polygon in the width direction in the to-be-detected projection data;

[0171] Compare the first pixel and the second pixel with a preset pixel threshold respectively, and determine whether both the first pixel and the second pixel are greater than the pixel threshold;

[0172] If so, continue to execute the step of identifying whether there is any undetected second object detection data;

[0173] If not, stop executing the step of identifying whether there is any undetected second object detection data.

[0174] Specifically, the step in which the first acquisition unit acquires the first object detection data includes:

[0175] Acquire the detection range of the current vehicle and first laser point cloud data including at least one first object to be detected within the detection range;

[0176] Based on map information, filter the first laser point cloud data to obtain preprocessed point cloud data including lanes;

[0177] Cluster and enclose the preprocessed point cloud data to obtain three-dimensional polygons corresponding to each of the first objects to be detected and first object detection data including each of the three-dimensional polygons.

[0178] For the specific definition of the device for identifying object undetected, reference can be made to the definition of the method for identifying object undetected in the foregoing text, which will not be elaborated here. Each unit in the above-mentioned device for identifying object undetected can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned units can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above-mentioned units.

[0179] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 3 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements a method for identifying object undetected. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0180] Those skilled in the art can understand, Figure 3The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0181] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0182] Obtain first object detection data, where the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected;

[0183] Obtain second object detection data, where the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle;

[0184] Determine whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data.

[0185] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0186] The step of determining whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data includes:

[0187] Project the first object detection data onto the second object detection data to obtain detection projection data to be detected;

[0188] Obtain the first object overlap degree according to the detection projection data to be detected and the second object detection data;

[0189] Compare the first object overlap degree with a preset overlap degree threshold to determine whether the first object overlap degree is greater than the overlap degree threshold;

[0190] If not, determine that there is a missed detection in the second object detection data;

[0191] If so, determine that there is no missed detection in the second object detection data.

[0192] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0193] The step of projecting the first object detection data onto the second object detection data to obtain the detection-projection data to be detected includes:

[0194] Obtain each of the first objects to be detected in the first object detection data, wherein the form of the first objects to be detected includes three-dimensional polygons;

[0195] Sample the three-dimensional polygons to obtain at least one key point, wherein the key points include the vertices of the three-dimensional polygon frame;

[0196] Project each of the key points into the camera coordinate system through the extrinsic camera parameters to obtain corresponding first projection points;

[0197] Project each of the first projection points onto the two-dimensional imaging plane of the camera through the intrinsic camera parameters to obtain corresponding second projection points;

[0198] Surround each of the second projection points to obtain a projection polygon;

[0199] Fuse the projection polygons of each of the three-dimensional polygons to obtain the detection-projection data to be detected.

[0200] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0201] Identify the first lidar point cloud data to determine whether the first lidar point cloud data includes a toll station;

[0202] If not, continue to execute the identification step of whether there is an undetected object in the second object detection data;

[0203] If so, obtain the distance between the toll station and the current vehicle, and determine whether the distance is less than a preset distance threshold;

[0204] If so, stop executing the identification step of whether there is an undetected object in the second object detection data;

[0205] If not, continue to execute the identification step of whether there is an undetected object in the second object detection data.

[0206] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0207] Obtain each of the first objects to be detected in the first object detection data including at least one three-dimensional polygon, wherein the form of the first objects to be detected includes three-dimensional polygons;

[0208] Obtain the first extreme point of each of the three-dimensional polygons in the first direction and the second extreme point in the second direction, where the first direction is used to indicate a direction perpendicular to the laser emission direction, and the second direction is used to indicate a direction parallel to the ground;

[0209] Expand the first extreme point according to a preset expansion distance to obtain a first expansion point, and expand the second extreme point according to the expansion distance to obtain a second expansion point;

[0210] Obtain the first reflection distance of the laser at the first expansion point and the second reflection distance of the laser at the second expansion point;

[0211] Compare the first reflection distance and the second reflection distance with the length of the three-dimensional polygon respectively, and determine whether both the first reflection distance and the second reflection distance are greater than the length of the three-dimensional polygon;

[0212] If so, determine that the three-dimensional polygon is not blocked, and continue to execute the identification step of whether there is a missed detection in the second object detection data;

[0213] If not, determine that the three-dimensional polygon is blocked, and stop executing the identification step of whether there is a missed detection in the second object detection data.

[0214] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0215] After the step of projecting the first object detection data onto the second object detection data to obtain the to-be-detected projection data, it includes:

[0216] Obtain first target object detection data, where the first target object detection data is obtained by detecting second laser point cloud data including at least one target object, and the distance from the target object to the current vehicle is less than the distance from the first object to be detected to the current vehicle;

[0217] Obtain second target object detection data, where the second target object detection data is obtained by detecting second camera image data including at least one of the target objects;

[0218] Project the first target object detection data onto the second target object detection data to obtain target projection data;

[0219] Obtain a second object overlap degree according to the to-be-detected projection data and the target projection data;

[0220] Compare the second object overlap degree with the overlap degree threshold, and determine whether the second object overlap degree is greater than the overlap degree threshold;

[0221] If so, stop executing the recognition step of whether there is undetected inspection in the second object detection data;

[0222] If not, continue to execute the recognition step of whether there is undetected inspection in the second object detection data.

[0223] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0224] After the step of projecting the first object detection data onto the second object detection data to obtain the to-be-detected projection data, it includes:

[0225] Obtain a first pixel and a second pixel of the to-be-detected projection data, wherein the first pixel is used to indicate the pixel in the length direction of the projection polygon in the to-be-detected projection data, and the second pixel is used to indicate the pixel in the width direction of the projection polygon in the to-be-detected projection data;

[0226] Compare the first pixel and the second pixel with a preset pixel threshold respectively to determine whether both the first pixel and the second pixel are greater than the pixel threshold;

[0227] If so, continue to execute the recognition step of whether there is undetected inspection in the second object detection data;

[0228] If not, stop executing the recognition step of whether there is undetected inspection in the second object detection data.

[0229] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0230] The step of obtaining the first object detection data includes:

[0231] Obtain the detection range of the current vehicle and the first lidar point cloud data including at least one first object to be detected within the detection range;

[0232] Based on the map information, filter the first lidar point cloud data to obtain the preprocessed point cloud data including lanes;

[0233] Cluster and enclose the preprocessed point cloud data to obtain the three-dimensional polygons corresponding to each of the first objects to be detected and the first object detection data including each of the three-dimensional polygons.

[0234] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0235] Obtain first object detection data, where the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected;

[0236] Obtain second object detection data, where the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle;

[0237] Determine whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data.

[0238] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0239] The step of determining whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data includes:

[0240] Project the first object detection data onto the second object detection data to obtain projection data to be detected;

[0241] Obtain the first object overlap degree according to the projection data to be detected and the second object detection data;

[0242] Compare the first object overlap degree with a preset overlap degree threshold to determine whether the first object overlap degree is greater than the overlap degree threshold;

[0243] If not, determine that there is a missed detection in the second object detection data;

[0244] If so, determine that there is no missed detection in the second object detection data.

[0245] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0246] The step of projecting the first object detection data onto the second object detection data to obtain projection data to be detected includes:

[0247] Obtain each of the first objects to be detected in the first object detection data, where the representation form of the first object to be detected includes a three-dimensional polygon;

[0248] Sample the three-dimensional polygon to obtain at least one key point, where the key point includes the vertex of the three-dimensional polygon box;

[0249] Project each of the key points into the camera coordinate system through the external camera parameters to obtain corresponding first projection points;

[0250] Project each of the first projection points into the two-dimensional imaging plane of the camera through the internal camera parameters to obtain corresponding second projection points;

[0251] Surround each of the second projection points to obtain a projection polygon;

[0252] Fuse the projection polygons of each of the three-dimensional polygons to obtain the projection data to be detected.

[0253] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0254] Identify the first lidar point cloud data to determine whether the first lidar point cloud data includes a toll station;

[0255] If not, continue to execute the identification step of whether there is a missed detection in the second object detection data;

[0256] If so, obtain the distance between the toll station and the current vehicle, and determine whether the distance is less than a preset distance threshold;

[0257] If so, stop executing the identification step of whether there is a missed detection in the second object detection data;

[0258] If not, continue to execute the identification step of whether there is a missed detection in the second object detection data.

[0259] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0260] Obtain each of the first objects to be detected included in the first object detection data, where the first objects to be detected include at least one three-dimensional polygon, and the representation form of the first objects to be detected includes three-dimensional polygons;

[0261] Obtain a first extreme point of each of the three-dimensional polygons in a first direction and a second extreme point in a second direction, where the first direction is used to indicate a direction perpendicular to the laser emission direction, and the second direction is used to indicate a direction parallel to the ground;

[0262] Expand the first extreme point according to a preset expansion distance to obtain a first expansion point, and expand the second extreme point according to the expansion distance to obtain a second expansion point;

[0263] Obtain a first reflection distance of the laser at the first expansion point and a second reflection distance of the laser at the second expansion point;

[0264] Compare the first reflection distance and the second reflection distance with the length of the three-dimensional polygon respectively to determine whether both the first reflection distance and the second reflection distance are greater than the length of the three-dimensional polygon;

[0265] If so, determine that the three-dimensional polygon is not blocked, and continue to execute the identification step of whether there is a missed detection in the second object detection data;

[0266] If not, determine that the three-dimensional polygon is blocked, and stop executing the identification step of whether there is a missed detection in the second object detection data.

[0267] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0268] After the step of projecting the first object detection data onto the second object detection data to obtain the to-be-detected projection data, it includes:

[0269] Obtain first target object detection data, where the first target object detection data is obtained by detecting second lidar point cloud data including at least one target object, and the distance from the target object to the current vehicle is less than the distance from the first object to be detected to the current vehicle;

[0270] Obtain second target object detection data, where the second target object detection data is obtained by detecting second camera image data including at least one of the target objects;

[0271] Project the first target object detection data onto the second target object detection data to obtain target projection data;

[0272] Obtain a second object overlap degree according to the to-be-detected projection data and the target projection data;

[0273] Compare the second object overlap degree with the overlap degree threshold to determine whether the second object overlap degree is greater than the overlap degree threshold;

[0274] If so, stop executing the identification step of whether there is a missed detection in the second object detection data;

[0275] If not, continue to execute the identification step of whether there is a missed detection in the second object detection data.

[0276] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0277] After the step of projecting the first object detection data onto the second object detection data to obtain the to-be-detected projection data, it includes:

[0278] Obtain a first pixel and a second pixel of the projection data to be detected, where the first pixel is used to indicate the pixel of the projection polygon in the length direction in the projection data to be detected, and the second pixel is used to indicate the pixel of the projection polygon in the width direction in the projection data to be detected;

[0279] Compare the first pixel and the second pixel with a preset pixel threshold respectively to determine whether both the first pixel and the second pixel are greater than the pixel threshold;

[0280] If so, continue to execute the recognition step of whether there is a missed detection in the second object detection data;

[0281] If not, stop executing the recognition step of whether there is a missed detection in the second object detection data.

[0282] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0283] The step of obtaining the first object detection data includes:

[0284] Obtain the detection range of the current vehicle and first lidar point cloud data including at least one first object to be detected within the detection range;

[0285] Based on the map information, filter the first lidar point cloud data to obtain preprocessed point cloud data including lanes;

[0286] Cluster and enclose the preprocessed point cloud data to obtain three-dimensional polygons corresponding to each of the first objects to be detected and first object detection data including each of the three-dimensional polygons.

[0287] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0288] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0289] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A method for identifying missed detection of an object, characterized in that, Including: Obtain first object detection data, where the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected, the first object detection data includes at least one of the first objects to be detected, and the representation form of the first object to be detected includes a three-dimensional polygon; Obtain second object detection data, where the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle; Obtain a first extreme point of each of the three-dimensional polygons in a first direction and a second extreme point in a second direction, where the first direction is used to indicate a direction perpendicular to the lidar emission direction, and the second direction is used to indicate a direction parallel to the ground; Expand the first extreme point according to a preset expansion distance to obtain a first expansion point, and expand the second extreme point according to the expansion distance to obtain a second expansion point; Obtain a first reflection distance of the lidar at the first expansion point and a second reflection distance of the lidar at the second expansion point; Compare the first reflection distance and the second reflection distance with the length of the three-dimensional polygon respectively, and determine whether both the first reflection distance and the second reflection distance are greater than the length of the three-dimensional polygon; if so, determine that the three-dimensional polygon is not blocked, and determine whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data; If not, determine that the three-dimensional polygon is blocked, and stop executing the recognition step of whether there is a missed detection in the second object detection data.

2. The method for identifying undetected objects according to claim 1, wherein The step of determining whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data includes: Project the first object detection data onto the second object detection data to obtain projection data to be detected; Obtain a first object overlap degree according to the projection data to be detected and the second object detection data; Compare the first object overlap degree with a preset overlap degree threshold, and determine whether the first object overlap degree is greater than the overlap degree threshold; If not, determine that there is a missed detection in the second object detection data; If so, determine that there is no missed detection in the second object detection data.

3. The method for identifying missed detection of an object according to claim 2, wherein The step of projecting the first object detection data onto the second object detection data to obtain projection data to be detected includes: Obtain each of the first objects to be detected in the first object detection data, where the representation form of the first object to be detected includes a three-dimensional polygon; Sample the three-dimensional polygon to obtain at least one key point, where the key point includes the vertex of the three-dimensional polygon box; Project each of the key points onto the camera coordinate system through the external camera parameters to obtain corresponding first projection points; Project each of the first projection points onto the two-dimensional imaging plane of the camera through the internal camera parameters to obtain corresponding second projection points; Surround each of the second projection points to obtain a projection polygon; Fuse the projection polygons of each of the three-dimensional polygons to obtain the projection data to be detected.

4. The method for identifying missed inspection of an object according to claim 1, wherein It further includes: Identify the first lidar point cloud data to determine whether the first lidar point cloud data includes a toll station; If not, continue to perform the identification step of whether there is a missed detection in the second object detection data; If so, obtain the distance between the toll station and the current vehicle, and determine whether the distance is less than a preset distance threshold; If so, stop performing the identification step of whether there is a missed detection in the second object detection data; If not, continue to perform the identification step of whether there is a missed detection in the second object detection data.

5. The method for identifying missed detection of an object according to claim 2, wherein After the step of projecting the first object detection data onto the second object detection data to obtain the projection data to be detected, it includes: Obtain first target object detection data, where the first target object detection data is obtained by detecting second lidar point cloud data including at least one target object, and the distance from the target object to the current vehicle is less than the distance from the first object to be detected to the current vehicle; Obtain second target object detection data, where the second target object detection data is obtained by detecting second camera image data including at least one of the target objects; Project the first target object detection data onto the second target object detection data to obtain target projection data; Obtain a second object overlap degree according to the projection data to be detected and the target projection data; Compare the second object overlap degree with an overlap degree threshold to determine whether the second object overlap degree is greater than the overlap degree threshold; If so, stop performing the identification step of whether there is a missed detection in the second object detection data; If not, continue to perform the identification step of whether there is a missed detection in the second object detection data.

6. The method for identifying missed detection of an object according to claim 3, characterized in that, After the step of projecting the first object detection data onto the second object detection data to obtain the projection data to be detected, it includes: Obtain a first pixel and a second pixel of the projection data to be detected, where the first pixel is used to indicate the pixel of the projection polygon in the length direction in the projection data to be detected, and the second pixel is used to indicate the pixel of the projection polygon in the width direction in the projection data to be detected; Compare the first pixel and the second pixel with a preset pixel threshold respectively to determine whether both the first pixel and the second pixel are greater than the pixel threshold; If so, continue to perform the identification step of whether there is a missed detection in the second object detection data; If not, stop performing the identification step of whether there is a missed detection in the second object detection data.

7. The method for identifying undetected objects according to claim 1, characterized in that, The step of obtaining the first object detection data includes: Obtain the detection range of the current vehicle and first lidar point cloud data including at least one first object to be detected within the detection range; Filter the first lidar point cloud data based on map information to obtain preprocessed point cloud data including lanes; Cluster and enclose the preprocessed point cloud data to obtain three-dimensional polygons corresponding to each of the first objects to be detected, and first object detection data including each of the three-dimensional polygons.

8. An identification device for missed detection of an object, characterized in that, The device includes: A first acquisition unit configured to acquire first object detection data, where the first object detection data is obtained by detecting first lidar point cloud data including at least one first object to be detected, the first object detection data includes at least one of the first objects to be detected, and the manifestation form of the first object to be detected includes a three-dimensional polygon; A second acquisition unit configured to acquire second object detection data, where the second object detection data is obtained by detecting first camera image data including at least one second object to be detected, and the distance from the first object to be detected to the current vehicle is equal to the distance from the second object to be detected to the current vehicle; A processing unit configured to: Obtain a first extreme point of each of the three-dimensional polygons in a first direction and a second extreme point in a second direction, where the first direction is used to indicate a direction perpendicular to the lidar emission direction, and the second direction is used to indicate a direction parallel to the ground; Expand the first extreme point according to a preset expansion distance to obtain a first expanded point, and expand the second extreme point according to the expansion distance to obtain a second expanded point; Obtain a first reflection distance of the lidar at the first expanded point and a second reflection distance of the lidar at the second expanded point; Compare the first reflection distance and the second reflection distance with the length of the three-dimensional polygon respectively to determine whether both the first reflection distance and the second reflection distance are greater than the length of the three-dimensional polygon; If so, determine that the three-dimensional polygon is not blocked, and determine whether there is a missed detection in the second object detection data according to the overlap degree between the first object detection data and the second object detection data; If not, determine that the three-dimensional polygon is blocked, and stop executing the identification step of whether there is a missed detection in the second object detection data.

9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for identifying missed detection of an object according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for identifying missed detection of an object according to any one of claims 1 to 7.

Images