A method, device and equipment for determining omission of object detection samples

Abstract

The embodiments of the present application disclose a method, device, and equipment for determining missing target detection samples. The method includes: obtaining a target image and a first boundary set; determining a pixel point set in which the pixel value change of the pixel points is greater than a preset threshold; determining the point cloud data of the pixel points in the pixel point set to obtain a point cloud set; clustering the point cloud data in the point cloud set to obtain a detection target set; determining the boundaries of each detection target in the detection target set to obtain a second boundary set; mapping the boundaries in the second boundary set to the target image to obtain a third boundary set; when the overlapping ratio of the region formed by the first boundaries in the third boundary set and the region formed by the second boundaries in the first boundary set is greater than the preset threshold, removing the first boundaries from the third boundary set; determining the result of missing target detection samples according to the boundaries in the third boundary set; for increasing the training data of the target detection model.

Description

Technical Field

[0001] The present invention relates to the field of computers, and particularly to a method, device, and equipment for determining missing target detection samples. Background Art

[0002] With the development of science and technology, in order to improve the automation level of driving, autonomous driving technology has come into people's view. Autonomous driving technology can automate the operation state of a vehicle through automated control, realizing the automation of the work performed by a driver. Implementing the autonomous driving function usually involves multiple detectors, and a camera is a commonly used detector in the process of autonomous driving for detecting an image. In the process of autonomous driving, the camera on the vehicle detects an image, and usually performs target detection on the image to provide a basis for decisions such as automatic obstacle avoidance for the vehicle.

[0003] Target detection is usually achieved through a target detection model, which is trained with existing training data. The training data includes the image to be detected, the detection targets in the image, and the labels corresponding to the detection targets. However, the results obtained by performing target detection on an image through the target detection model are not necessarily accurate, and there may be a phenomenon of missing samples, that is, for one or more detection targets actually existing in the image, the detection results of these one or more detection targets are not obtained after target detection.

[0004] Currently, in order to improve the target detection ability of the target detection model and reduce the occurrence of missing target detection samples, usually, a person obtains these one or more detection targets and corresponding labels in the image, and adds the obtained results to the training data to train the target detection model. It is inefficient for a person to obtain these one or more detection targets from the image data. Therefore, there is an urgent need for a method for determining missing target detection samples to improve the efficiency of determining missing target detection samples. Summary of the Invention

[0005] In view of this, the present application provides a method, device, and equipment for determining missing target detection samples to increase the training data of the target detection model.

[0006] In a first aspect, the present application provides a method for determining missing target detection samples, the method including:

[0007] Obtain a target image and a first boundary set, where the boundaries in the first boundary set are the boundaries of the detection targets in the target image;

[0008] Determine a pixel point set, where the pixel points in the pixel point set are the pixel points in the target image, and the pixel value change of the pixel points in the pixel point set is greater than a preset threshold;

[0009] Determine the point cloud data of the pixels in the set of pixel points to obtain a point cloud set, where the point cloud data of each pixel point includes the three-dimensional position coordinates of the measured object corresponding to the pixel point in the world coordinate system;

[0010] Cluster the point cloud data in the point cloud set to obtain a set of detection targets, and the detection targets in the set of detection targets correspond one-to-one with the classes of the point cloud data in the point cloud set;

[0011] Determine the boundaries of each detection target in the set of detection targets to obtain a second set of boundaries;

[0012] Map the boundaries in the second set of boundaries to the target image to obtain a third set of boundaries;

[0013] When the overlapping ratio of the region formed by the first boundary in the third set of boundaries and the region formed by the second boundary in the first set of boundaries is greater than a preset threshold, remove the first boundary from the third set of boundaries;

[0014] Determine the result of the omission of the target detection sample according to the boundaries in the third set of boundaries.

[0015] Determine the boundary of the detection target in the target image according to the change of the pixel value, and obtain the point cloud data of the boundary. Cluster the point cloud data to obtain the three-dimensional boundary of the detection target and project it onto the target image to obtain the two-dimensional boundary of the detection target. Then compare the two-dimensional boundary of the detection target with the boundary in the first set of boundaries obtained by the original target detection model, and determine whether there is an omission in the boundary of the detection target obtained by the original target detection model according to the comparison result. Determine whether the target detection sample is omitted in an automated manner to improve efficiency.

[0016] In a possible implementation manner, before clustering the point cloud data in the point cloud set to obtain a set of detection targets, and the detection targets in the set of detection targets correspond one-to-one with the classes of the point cloud data, it further includes:

[0017] Determine the target area;

[0018] Determine the target point cloud data in the point cloud set according to the target area to obtain a target point cloud set;

[0019] Clustering the point cloud data in the point cloud set to obtain a set of detection targets, and the detection targets in the set of detection targets correspond one-to-one with the classes of the point cloud data, specifically includes:

[0020] Cluster the point cloud data in the target point cloud set to obtain a set of detection targets, and the detection targets in the set of detection targets correspond one-to-one with the classes of the point cloud data in the target point cloud set.

[0021] In a possible implementation manner, determining the target area specifically includes:

[0022] The position of the detector when the target image is obtained;

[0023] Determine the target area according to the detector position and the map data, where the map data includes road position information, road shape information, and road size information.

[0024] In a possible implementation, the determination of the pixel point set, where the pixel points in the pixel point set are the pixel points in the target image, and the pixel value change of the pixel points in the pixel point set is greater than a preset threshold, specifically includes:

[0025] Determine the pixel points in the target image whose absolute value of the pixel value gradient is greater than a second preset threshold to obtain the pixel point set.

[0026] In a second aspect, the present application provides a device for determining the omission of target detection samples, and the device includes:

[0027] A first image processing unit, configured to obtain a target image and a first boundary set, where the boundaries in the first boundary set are the boundaries of the detection target in the target image; determine a pixel point set, where the pixel points in the pixel point set are the pixel points in the target image, and the pixel value change of the pixel points in the pixel point set is greater than a preset threshold; determine the point cloud data of the pixel points in the pixel point set to obtain a point cloud set, and the point cloud data of each pixel point includes the three-dimensional position coordinates of the measured object corresponding to the pixel point in the world coordinate system;

[0028] A point cloud data processing unit, configured to cluster the point cloud data in the point cloud set to obtain a detection target set, where the detection targets in the detection target set correspond one-to-one with the classes of the point cloud data in the point cloud set; determine the boundaries of each detection target in the detection target set to obtain a second boundary set; map the boundaries in the second boundary set to the target image to obtain a third boundary set;

[0029] A second image processing unit, configured to remove the first boundary in the third boundary set when the overlapping ratio of the area formed by the first boundary in the third boundary set and the area formed by the second boundary in the first boundary set is greater than a preset threshold; determine the result of the omission of the target detection sample according to the boundaries in the third boundary set.

[0030] In a possible implementation, the point cloud data processing unit is further configured to: determine the target area; determine the target point cloud data in the point cloud set according to the target area to obtain a target point cloud set;

[0031] The point cloud data processing unit is specifically configured to: cluster the point cloud data in the target point cloud set to obtain a detection target set, where the detection targets in the detection target set correspond one-to-one with the classes of the point cloud data in the target point cloud set.

[0032] In a possible implementation, the point cloud data processing unit is specifically configured to:

[0033] Obtain the detector position when the target image is acquired;

[0034] Determine a target area according to the detector position and the map data, where the map data includes road position information, road shape information, and road size information.

[0035] In a possible implementation, the first image processing unit is specifically configured to determine pixel points in the target image whose absolute value of the gradient of the pixel value is greater than a second preset threshold, and obtain a set of pixel points.

[0036] In a third aspect, the present application provides a device for determining omission of target detection samples. The device includes a processor and a memory. The memory stores code, and the processor is configured to call the code stored in the memory to execute the method as described in any one of the above.

[0037] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, and the computer program is used to execute the method as described in any one of the above. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flowchart of the method for determining omission of target detection samples provided by an embodiment of the present application;

[0039] Figure 2 is a schematic structural diagram of the device for determining omission of target detection samples provided by an embodiment of the present application;

[0040] Figure 3 is a schematic structural diagram of the device for determining omission of target detection samples provided by an embodiment of the present application. DETAILED DESCRIPTION

[0041] Currently, in order to improve the target detection ability of the target detection model and reduce the occurrence of omission of target detection samples, usually one or more detection targets and corresponding labels are obtained from image data manually, and the obtained results are added to the training data to train the target detection model. Obtaining one or more detection targets from image data manually has low efficiency. Therefore, there is an urgent need for a method for determining omission of target detection samples to improve the efficiency of determining omission of target detection samples.

[0042] Based on this, in the embodiments of the present application provided by the applicant, first, an image and a first boundary set of the image are obtained; according to the change in the gray value of the pixel points in the image, the boundary of the detection target in the image is determined, and the point cloud data corresponding to the boundary of the detection target is obtained. The point cloud data is clustered to obtain a three-dimensional boundary, and then the three-dimensional boundary is mapped into a two-dimensional image to obtain the boundary of the detection target obtained from the point cloud data, and a third boundary set is obtained; the first boundary set and the third boundary set are compared to obtain the result of missing target detection samples.

[0043] To facilitate the understanding of the technical solutions provided by the embodiments of the present application, first, the common application scenarios of the embodiments of the present application are introduced.

[0044] The method for determining missing target detection samples in the present application can be applied to the generation of training data for the target detection model in the autonomous driving scenario to optimize the target detection model and improve the accuracy of target detection.

[0045] The results obtained by performing target detection on an image through a target detection model are not necessarily accurate, and there may be a phenomenon of missing samples. That is, for one or more detection targets actually existing in the image, the detection results of these one or more detection targets are not obtained after target detection.

[0046] Currently, in order to improve the target detection ability of the target detection model and reduce the occurrence of missing target detection samples, usually one or more detection targets and corresponding labels are obtained manually in the image, and the obtained results are added to the training data to train the target detection model. The efficiency of obtaining one or more detection targets manually from image data is relatively low. Therefore, there is an urgent need for a method for determining missing target detection samples to improve the efficiency of determining missing target detection samples.

[0047] After determining that there are missing samples in target detection, the missing samples (detection targets) can be labeled by, for example, manual annotation, and then the missing samples and the corresponding annotation results are added to the original results to increase the training data of the target detection model.

[0048] To facilitate the understanding of the technical solutions provided by the embodiments of the present application, the following describes a method, device, and equipment for determining missing target detection samples provided by the embodiments of the present application with reference to the accompanying drawings.

[0049] Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Based on the embodiments in the present application, other embodiments obtained by those skilled in the art without making creative contributions all fall within the protection scope of the present application.

[0050] In the claims, specification, and drawings of the present application, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion.

[0051] The present application provides a method for determining omission of target detection samples.

[0052] Please refer to Figure 1 , Figure 1 which is a flowchart of the method for determining omission of target detection samples provided by an embodiment of the present application.

[0053] As Figure 1 shown, the method for determining omission of target detection samples in an embodiment of the present application includes S101 - S107.

[0054] S101. Obtain a target image and a first boundary set, where the boundaries in the first boundary set are the boundaries of detection targets in the target image.

[0055] The target image refers to the detection result of a camera.

[0056] The target image represents the object to be measured in the form of pixels, and includes the two-dimensional position coordinates of the object to be measured in the pixel coordinate system and the pixel values of the corresponding pixel points of the object to be measured.

[0057] The target image includes one or more detection targets, and the detection target is one or more objects to be measured in the image, such as a person, a vehicle, etc.

[0058] The process of target detection is the process of determining one or more detection targets from the image.

[0059] The first boundary set is obtained by performing target detection on the target image, and the first boundary set includes one or more boundaries.

[0060] The boundary of the detection target refers to the demarcation position between the area of the detection target in the image and other parts according to the target detection result.

[0061] It can be understood that the detection targets corresponding to the boundaries in the first boundary set are obtained by the target detection process, and are not necessarily the real detection targets in the target image, nor are they necessarily all the detection targets in the target image.

[0062] S102. Determine a pixel point set, where the pixel points in the pixel point set are the pixel points in the target image, and the pixel value changes of the pixel points in the pixel point set are greater than a preset threshold.

[0063] The pixel point set includes one or more pixel points, and usually includes multiple pixel points.

[0064] There are usually multiple pixel points with large changes in pixel values in the target image, and they are the boundaries of the detection target, that is, the contours of the detection target.

[0065] S103. Determine the point cloud data of the pixel points in the pixel point set to obtain a point cloud set. The point cloud data of each pixel point includes the three-dimensional position coordinates of the corresponding measured object in the world coordinate system.

[0066] Each pixel point in the pixel point set corresponds to point cloud data.

[0067] Each pixel point in the pixel point set corresponds to point cloud data.

[0068] The pixel points in the target image correspond to the measured object. The position of a pixel point in the target image is the position of the corresponding measured object in the pixel coordinate system, represented in the form of two-dimensional position coordinates.

[0069] That is, the two-dimensional position coordinates of the measured object in the target image coordinate system can be determined according to the target image.

[0070] S103 is also the process of obtaining the three-dimensional position coordinates of the measured object in the world coordinate system from the two-dimensional position coordinates of the measured object in the pixel coordinate system.

[0071] S104. Cluster the point cloud data in the point cloud set to obtain a detection target set. The detection targets in the detection target set correspond one-to-one with the classes of the point cloud data.

[0072] The point cloud data of the same class is the point cloud data of a detection target in the detection target set in the world coordinate system.

[0073] The point cloud data of each pixel point includes the three-dimensional position coordinates of the corresponding measured object in the world coordinate system.

[0074] S105. Determine the boundaries of each detection target in the detection target set to obtain a second boundary set.

[0075] The detection targets in the detection target set are three-dimensional detection targets in the world coordinate system, and the boundaries in the second boundary set are three-dimensional boundaries in the world coordinate system.

[0076] S106. Map the boundaries in the second boundary set to the target image to obtain a third boundary set.

[0077] The boundaries in the second boundary set are the three-dimensional boundaries of the object. After being mapped to a two-dimensional image, the boundaries in the third boundary set are two-dimensional boundaries.

[0078] S107. When the overlapping ratio of the region formed by the first boundaries in the third boundary set and the region formed by the second boundaries in the first boundary set is greater than a preset ratio threshold, remove the first boundary from the third boundary set.

[0079] The boundaries in the third boundary set are the detection results obtained from the point cloud data; the boundaries in the first detection target set are the detection results obtained from S101, that is, comparing the detection results obtained by the two methods.

[0080] S108. Determine the results of the missing detection targets of the target detection samples according to the boundaries in the third boundary set.

[0081] The boundaries in the third boundary set can determine the detection targets obtained from the point cloud data, and the boundaries in the first boundary set can determine the detection targets determined by the first target detection results. The results obtained by S108 are the detection targets obtained from the point cloud data but not obtained by the first target detection results, that is, the missing detection targets.

[0082] When the third boundary set does not contain any boundaries, the result of the missing target detection samples is that there is no missing sample; when the third boundary set contains at least one boundary, the result of the missing target detection samples is that there is a missing sample, and the missing sample is the boundary contained in the third boundary set.

[0083] Determine the boundaries of the detection targets in the target image according to the change of pixel values, obtain the point cloud data of the boundaries, cluster the point cloud data to obtain the three-dimensional boundaries of the detection targets and project them into the target image to obtain the two-dimensional boundaries of the detection targets, then compare the two-dimensional boundaries of the detection targets with the boundaries in the first boundary set obtained by the original target detection model, and determine whether there are missing boundaries of the detection targets obtained by the original target detection model according to the comparison results, and determine whether there are missing target detection samples in an automated manner, improving efficiency.

[0084] The embodiment of the present application also provides another method for determining missing target detection samples, and this method is applied to the autonomous driving scenario.

[0085] The method in this embodiment includes S201 - S209.

[0086] S201. Obtain the target image detected by the camera on the autonomous driving vehicle, and the first target detection result of the target image, where the first target detection result includes the first boundary set.

[0087] In a possible implementation manner, the bounding box of the boundary in the first boundary set can also be determined. That is, determine that the shape of the boundary is a rectangle.

[0088] The first boundary set contains one or more boundaries, and each boundary corresponds to a bounding box.

[0089] In a possible implementation, the bounding box is displayed in the form of a rectangular box in the pixel coordinate system.

[0090] S202. Determine a set of pixel points. The pixel points in the set of pixel points are the pixel points in the target image, and the change in the pixel values of the pixel points in the set of pixel points is greater than a preset threshold.

[0091] In a possible implementation, the change in the gray value of a pixel point is obtained by determining the change in the gray value of the pixel point within a preset neighborhood.

[0092] In a possible implementation, the change in the gray value of a pixel point is obtained by determining the absolute value of the gradient of the gray value of the pixel point.

[0093] Determine multiple pixel points in the target image whose absolute value of the gradient of the gray value is greater than a second preset threshold.

[0094] When the absolute value of the gradient of the gray value of a pixel point is large, the change in the gray value of the pixel point is large; when the absolute value of the gradient of the gray value of a pixel point is small, the change in the gray value of the pixel point is small.

[0095] Furthermore, the gradient operator can be set for the preset neighborhood of the pixels in the target image by using the change rule of the first-order or second-order derivative near the edge, and the absolute value of the gradient of the gray value of the pixel point can be determined.

[0096] S203. Determine the point cloud data of the pixel points in the set of pixel points to obtain a point cloud set. The point cloud data of each pixel point includes the three-dimensional position coordinates of the measured object corresponding to the pixel point in the world coordinate system.

[0097] In a possible implementation, first, the three-dimensional position coordinates of the measured object in the vehicle coordinate system are obtained from the two-dimensional position coordinates of the measured object in the pixel coordinate system, and then, based on the position of the vehicle in the world coordinate system, the three-dimensional position coordinates of the measured object in the world coordinate system are obtained, that is, the above-mentioned point cloud set.

[0098] In a possible implementation, to determine the point cloud data of the pixel points in the set of pixel points to obtain a point cloud set, it can be achieved through the following method:

[0099] Obtain a set of images detected by the camera within a preset time period. The set of images includes the above-mentioned target image;

[0100] Based on the images in the set of images, obtain the point cloud data of the pixel points in the above-mentioned set of pixel points.

[0101] That is, point cloud data is constructed from multiple frames of images captured by a single camera on the vehicle.

[0102] In a possible implementation, a second set of images at a target detection moment is obtained. The target detection moment is the moment when the camera captures the above-mentioned target image. The second set of images includes the above-mentioned target image. The images in the second set of images are obtained by cameras at different positions on the vehicle. The objects to be measured in the images in the second set of images all include the objects corresponding to the pixels in the above-mentioned pixel point set.

[0103] That is, point cloud data is constructed from multiple frames of images obtained by multiple cameras on the vehicle.

[0104] S204. Obtain the vehicle position when the above-mentioned target image is obtained, determine the target area according to the vehicle position, and determine the target point cloud data in the point cloud set according to the target area to obtain the target point cloud set.

[0105] The vehicle position refers to the position of the vehicle in the world coordinate system.

[0106] Since the relative positions of the vehicle and the camera are fixed, in some possible cases, the camera position when the above-mentioned target image is obtained is obtained, and the target area is determined according to the camera position. Here, the camera position, that is, the detector position, is the position of the camera in the world coordinate system.

[0107] The vehicle position and the camera position can be obtained through the vehicle's positioning system.

[0108] The target point cloud data is the point cloud data in the target point cloud set.

[0109] An image usually contains multiple objects to be measured. It is possible that not all of the objects to be measured are necessary, or not all of the objects to be measured need to be detected during target detection.

[0110] For example, when the vehicle is driving autonomously, the target detection process usually focuses on the targets on the road, such as people, vehicles, etc.

[0111] Extract or filter out the data within the target area to reduce the amount of data to be processed, thereby improving the detection speed.

[0112] Since target detection is usually used to provide a basis for decisions such as automatic obstacle avoidance for the vehicle, so as to achieve autonomous driving or improve the safety of autonomous driving, usually the selection of the target area is related to the vehicle position.

[0113] For example, the target area is the road where the vehicle is located.

[0114] To improve the safety of autonomous driving, the target area can include not only the road where the vehicle is located, but also other roads. For example, it includes some or all of the roads contained in the target image.

[0115] Furthermore, the size of the target area can be determined according to actual needs. Since the data source for target detection is the target image detected by the camera, the target area usually does not include areas outside the area corresponding to the target image.

[0116] Furthermore, the target area is used to screen the point cloud data in the point cloud set to obtain target point cloud data, thereby obtaining a target point cloud set, and the target area is a three-dimensional space area.

[0117] In S204, the measured target corresponding to the point cloud data in the obtained target point cloud set is located in the target area.

[0118] In some possible implementation manners, determining the target area according to the vehicle position may specifically include:

[0119] Determine the target area according to the vehicle position and map data.

[0120] In the field of autonomous driving, a high-precision map is a commonly used map tool. The high-precision map can provide three-dimensional position information of roads, etc. in the world coordinate system, and specifically can include the three-dimensional position of the road, the shape of the road, the size of the road, etc.

[0121] Since target detection focuses on the road where the vehicle is located, therefore, according to the vehicle position / camera position, the road where the vehicle is located can be determined by using the high-precision map.

[0122] S205. Cluster the point cloud data in the target point cloud set to obtain a detection target set, and the detection targets in the detection target set correspond one-to-one with the classes of the point cloud data.

[0123] The point cloud data of the same class is the point cloud data of a detection target in the world coordinate system in the target set.

[0124] This embodiment provides an implementation manner for clustering the target point cloud data.

[0125] The point cloud data of each pixel point includes the three-dimensional position coordinates of the measured object corresponding to the pixel point in the world coordinate system.

[0126] Cluster the point cloud data in the target point cloud set according to the distance. For example, clustering is implemented based on the K-Means algorithm.

[0127] After clustering the point cloud data in the target point cloud set, multiple classes are obtained, and each class corresponds to a detection target in the detection set.

[0128] Since the point cloud data in the target point cloud set is the point cloud data of the target area, the detection targets obtained at this time are the detection targets within the target area.

[0129] S206. Determine the boundaries of each detection target in the detection target set to obtain a second boundary set.

[0130] The detection targets in the detection target set are three-dimensional detection targets in the world coordinate system, and the boundaries in the second boundary set are three-dimensional boundaries in the world coordinate system.

[0131] S207. Map each boundary in the second boundary set to the target image to obtain a third boundary set.

[0132] In a possible implementation, the bounding box of the boundary in the third boundary set can also be determined.

[0133] In a possible implementation, the bounding box of the boundary in the third boundary set can also be determined. That is, determine that the shape of the boundary is a rectangle.

[0134] The third boundary set contains one or more boundaries, and each boundary corresponds to a bounding box.

[0135] In a possible implementation, the bounding box is displayed in the form of a rectangular box in the pixel coordinate system.

[0136] S208. When the overlapping ratio of the region formed by the first boundary in the third boundary set and the region formed by the second boundary in the first boundary set is greater than a preset ratio threshold, remove the first boundary from the third boundary set.

[0137] The boundaries in the third boundary set are the detection results obtained from the point cloud data; the boundaries in the first detection target set are the detection results obtained from S201, that is, compare the detection results obtained by the two methods.

[0138] S209. Determine the results missed by the target detection samples according to the boundaries in the third boundary set.

[0139] The boundaries in the third boundary set can determine the detection targets obtained from the point cloud data, and the boundaries in the first boundary set can determine the detection targets determined by the first target detection result. The result obtained in S209 is the detection target obtained from the point cloud data but not obtained by the first target detection result, that is, the missed detection target, and it is determined whether the target detection sample is missed in an automated manner to improve efficiency.

[0140] Further, after S209, it is also possible to determine the labels of the missing detection targets, and add the target image, the missing detection targets, and the labels of the missing detection targets to the training data to train the target detection model and improve the accuracy of target detection.

[0141] The embodiment of the present application also provides a device for determining missing target detection samples.

[0142] Please refer to Figure 2 , Figure 2 which is a schematic structural diagram of the device for determining missing target detection samples provided by the embodiment of the present application.

[0143] As Figure 2 shown, the device for determining missing target detection samples in the embodiment of the present application includes:

[0144] A first image processing unit 201, configured to obtain a target image and a first boundary set, where the boundaries in the first boundary set are the boundaries of detection targets in the target image; determine a pixel point set, where the pixel points in the pixel point set are pixel points in the target image, and the pixel value change of the pixel points in the pixel point set is greater than a preset threshold; determine the point cloud data of the pixel points in the pixel point set to obtain a point cloud set, and the point cloud data of each pixel point includes the three-dimensional position coordinates of the measured object corresponding to the pixel point in the world coordinate system;

[0145] A point cloud data processing unit 202, configured to cluster the point cloud data in the point cloud set to obtain a detection target set, where the detection targets in the detection target set correspond one-to-one with the classes of the point cloud data in the point cloud set; determine the boundaries of each detection target in the detection target set to obtain a second boundary set; map the boundaries in the second boundary set to the target image to obtain a third boundary set;

[0146] A second image processing unit 203, configured to remove the first boundary from the third boundary set when the overlapping ratio of the region formed by the first boundaries in the third boundary set and the region formed by the second boundaries in the first boundary set is greater than a preset threshold; determine the result of missing target detection samples according to the boundaries in the third boundary set.

[0147] The units included in the above device for determining missing target detection samples can achieve the same technical effects as the method for determining missing target detection samples in the above embodiments. To avoid repetition, details are not described here again.

[0148] The embodiment of the present application also provides a device for determining missing target detection samples.

[0149] Please refer to Figure 3 , Figure 3 which is a schematic structural diagram of the device for determining missing target detection samples provided by the embodiment of the present application.

[0150] As Figure 3 shown, the device for determining omission of target detection samples in the embodiments of the present application includes a processor 301 and a memory 302. Among them, the memory 302 stores code, and the processor 301 is configured to call the code stored in the memory to execute the method described in any one of the above.

[0151] The units included in the above device for determining omission of target detection samples can achieve the same technical effects as the method for determining omission of target detection samples in the above embodiments. To avoid repetition, details are not described here again.

[0152] In the embodiments of the present application, a computer-readable storage medium is further provided. The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method for determining omission of target detection samples described above and can achieve the same technical effects. To avoid repetition, details are not described here again. Among them, the computer-readable storage medium includes, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0153] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining omission of object detection samples, characterized in that, The method includes: Obtaining a target image and a first boundary set, where the boundaries in the first boundary set are the boundaries of the detection target in the target image; Determining a pixel point set, where the pixel points in the pixel point set are the pixel points in the target image, and the pixel value change of the pixel points in the pixel point set is greater than a preset threshold; Determining the point cloud data of the pixel points in the pixel point set to obtain a point cloud set, and the point cloud data of each pixel point includes the three-dimensional position coordinates of the measured object corresponding to the pixel point in the world coordinate system; Clustering the point cloud data in the point cloud set to obtain a detection target set, and the detection targets in the detection target set correspond one-to-one with the classes of the point cloud data in the point cloud set; Determining the boundaries of each detection target in the detection target set to obtain a second boundary set, where the boundaries in the second boundary set are three-dimensional boundaries; Mapping the boundaries in the second boundary set to the target image to obtain a third boundary set, where the boundaries in the third boundary set are two-dimensional boundaries; When the overlapping ratio of the region formed by the first boundaries in the third boundary set and the region formed by the second boundaries in the first boundary set is greater than a preset threshold, removing the first boundaries in the third boundary set; Determining the result of missing target detection samples according to the boundaries in the third boundary set.

2. The method according to claim 1, wherein Before clustering the point cloud data in the point cloud set to obtain a detection target set, where the detection targets in the detection target set correspond one-to-one with the classes of the point cloud data, it further includes: Determining a target region; Determining the target point cloud data in the point cloud set according to the target region to obtain a target point cloud set; Clustering the point cloud data in the point cloud set to obtain a detection target set, where the detection targets in the detection target set correspond one-to-one with the classes of the point cloud data, specifically includes: Clustering the point cloud data in the target point cloud set to obtain a detection target set, and the detection targets in the detection target set correspond one-to-one with the classes of the point cloud data in the target point cloud set.

3. The method according to claim 2, wherein The determining the target region specifically includes: Obtaining the detector position when obtaining the target image; Determining the target region according to the detector position and map data, where the map data includes road position information, road shape information, and road size information.

4. The method according to claim 1, wherein The determining the pixel point set, where the pixel points in the pixel point set are the pixel points in the target image, and the pixel value change of the pixel points in the pixel point set is greater than a preset threshold, specifically includes: Determining the pixel points in the target image whose absolute value of the pixel value gradient is greater than a second preset threshold to obtain a pixel point set.

5. An apparatus for determining omission of object detection samples, characterized in that, The device includes: A first image processing unit, configured to obtain a target image and a first boundary set, where the boundaries in the first boundary set are the boundaries of the detection target in the target image; determine a pixel point set, where the pixel points in the pixel point set are the pixel points in the target image, and the pixel value change of the pixel points in the pixel point set is greater than a preset threshold; determine the point cloud data of the pixel points in the pixel point set to obtain a point cloud set, and the point cloud data of each pixel point includes the three-dimensional position coordinates of the measured object corresponding to the pixel point in the world coordinate system; A point cloud data processing unit is configured to cluster the point cloud data in a point cloud set to obtain a detection target set, where the detection targets in the detection target set correspond one-to-one with the classes of the point cloud data in the point cloud set; determine the boundaries of the respective detection targets in the detection target set to obtain a second boundary set, where the boundaries in the second boundary set are three-dimensional boundaries; map the boundaries in the second boundary set to a target image to obtain a third boundary set, where the boundaries in the third boundary set are two-dimensional boundaries. A second image processing unit is configured to, when the overlapping ratio of the region formed by a first boundary in the third boundary set and the region formed by a second boundary in a first boundary set is greater than a preset threshold, remove the first boundary from the third boundary set; determine the result of omission of target detection samples according to the boundaries in the third boundary set.

6. The device according to claim 5, wherein The point cloud data processing unit is further configured to: determine a target region; determine target point cloud data in the point cloud set according to the target region to obtain a target point cloud set. The point cloud data processing unit is specifically configured to: cluster the point cloud data in the target point cloud set to obtain a detection target set, where the detection targets in the detection target set correspond one-to-one with the classes of the point cloud data in the target point cloud set.

7. The device according to claim 6, characterized in that, The point cloud data processing unit is specifically configured to: Obtain the position of the detector when the target image is obtained. Determine the target region according to the detector position and map data, where the map data includes road position information, road shape information, and road size information.

8. The device according to claim 5, characterized in that, The first image processing unit is specifically configured to determine the pixel points in the target image whose absolute value of the gradient of the pixel value is greater than a second preset threshold to obtain a pixel point set.

9. A device for determining omission of object detection samples, characterized in that, The device includes a processor and a memory. Among them, the memory stores code, and the processor is configured to call the code stored in the memory to execute the method according to any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is configured to store a computer program, and the computer program is configured to execute the method according to any one of claims 1 to 4.

Images