A method and apparatus for optimizing a bounding box of a three-dimensional detected object

Abstract

The application relates to the technical field of target detection, and discloses a three-dimensional object candidate frame optimization method and device. The application is based on the three-dimensional characteristics of a target detection object, uses a target neural network to predict the position confidence of multiple candidate frames of the target detection object, optimizes the multiple candidate frames of the target detection object, obtains the positioning certainty probability value of the multiple candidate frames, obtains the target detection area of the target detection object based on the positioning certainty probability value of the multiple candidate frames, suppresses the position confidence of the multiple candidate frames in the target detection area, obtains the optimal candidate frame of the target detection object, and finally, the application is not only beneficial to accurately removing the redundant candidate frames of the target detection object, but also beneficial to accurately positioning the target detection object. When facing a complex or dynamic environment, the application can still generate stable detection results, thereby playing a key role in applications such as automatic driving and robot navigation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to a method and device for optimizing a candidate box of a three-dimensional detected object. BACKGROUND

[0002] Target detection not only needs to detect the objects existing on an image and give the corresponding object category, but also needs to give the position of the object in the form of a minimum enclosing box (candidate box). With the development of target detection technology, three-dimensional target detection has been widely concerned and applied in the field of image detection.

[0003] In related technologies, because the three-dimensional detected object is easily affected by occlusion or sensor noise in the three-dimensional target detection process, too many redundant candidate boxes are generated during detection, which finally easily leads to inaccurate detection results of the three-dimensional detected object. SUMMARY

[0004] Therefore, the present application provides a method and device for optimizing a candidate box of a three-dimensional detected object to solve the problem of inaccurate detection results of the three-dimensional detected object caused by too many redundant candidate boxes in the three-dimensional target detection process.

[0005] According to a first aspect, the present application provides a method for optimizing a candidate box of a three-dimensional detected object, which comprises:

[0006] Based on the three-dimensional features of the target detected object, the position confidence of a plurality of candidate boxes of the target detected object is predicted by using a target neural network, wherein the target neural network is trained in advance;

[0007] Based on the position confidence of the plurality of candidate boxes of the target detected object, the plurality of candidate boxes of the target detected object are processed in turn by using a segmentation processing method and a normalization processing method to obtain the positioning certainty probability value of the plurality of candidate boxes;

[0008] Based on the positioning certainty probability value of the plurality of candidate boxes, the target detection region of the target detected object is obtained, and the position confidence of the plurality of candidate boxes in the target detection region is obtained;

[0009] The position confidence of the plurality of candidate boxes in the target detection region is suppressed to obtain the optimal candidate box of the target detected object.

[0010] The embodiment of the present disclosure is based on the three-dimensional features of the target detection object, and a target neural network is used to predict the position confidence of a plurality of candidate boxes of the target detection object, wherein the target neural network is trained in advance; based on the position confidence of the plurality of candidate boxes of the target detection object, the plurality of candidate boxes of the target detection object are processed in turn through a segmentation processing mode and a normalization processing mode to obtain a positioning certainty probability value of the plurality of candidate boxes; based on the positioning certainty probability value of the plurality of candidate boxes, a target detection region of the target detection object is obtained, and the position confidence of the plurality of candidate boxes in the target detection region is obtained; the position confidence of the plurality of candidate boxes in the target detection region is suppressed to obtain an optimal candidate box of the target detection object, and finally the embodiment of the present disclosure is not only conducive to accurately removing the redundant candidate boxes of the target detection object, but also conducive to accurately positioning the target detection object.

[0011] In some optional embodiments, based on the positioning certainty probability value of the plurality of candidate boxes, the target detection region of the target detection object is obtained, including:

[0012] determining whether the positioning certainty probability value of the target candidate box of the target detection object is greater than a preset threshold, wherein the target candidate box is any candidate box in the plurality of candidate boxes;

[0013] if the positioning certainty probability value of the target candidate box of the target detection object is greater than the preset threshold, it is determined that there is a voxel in the region where the target candidate box is located;

[0014] if the positioning certainty probability value of the target candidate box of the target detection object is not greater than the preset threshold, the target candidate box is deleted;

[0015] all regions with voxels are taken as the target detection region of the target detection object.

[0016] The embodiment of the present disclosure is conducive to accurately removing redundant candidate boxes and accurately locking the target detection region of the target detection object through the above-mentioned manner.

[0017] In some optional embodiments, the candidate box optimization method for the three-dimensional detection object in the embodiment of the present disclosure further includes:

[0018] based on the optimal candidate box of the target detection object, the target detection object in the target detection region is positioned.

[0019] On the basis of removing the redundant candidate boxes of the target detection object to obtain the optimal candidate box, the embodiment of the present disclosure can further quickly locate the target detection object in the target detection region, so that even if the target detection object in the embodiment of the present disclosure is affected by factors such as occlusion and sensor noise, the potential blurred region can be more accurately detected, and finally the positioning accuracy is improved.

[0020] In some optional embodiments, the plurality of candidate boxes of the target detection object are optimized by a segmentation processing method, including:

[0021] Based on the position confidence of the plurality of candidate boxes of the target detection object, the plurality of candidate boxes of the target detection object that are not connected to each other are segmented for the first time by using a preset segmentation algorithm;

[0022] The target detection object that is connected to each other is obtained from the first segmentation result of the plurality of candidate boxes, and the target detection object that is connected to each other is segmented for the second time by using a neighborhood maximum suppression method.

[0023] The embodiments of the present disclosure process the position confidence of the plurality of candidate boxes by using the segmentation processing method, so as to delete the redundant candidate boxes of the target detection object, and further achieve accurate positioning of the target detection object.

[0024] In some optional embodiments, the plurality of candidate boxes of the target detection object are processed by a normalization processing method, to obtain a positioning certainty probability value of the plurality of candidate boxes, including:

[0025] Based on the position confidence of the plurality of candidate boxes of the target detection object, a total confidence sum of the target detection object is calculated;

[0026] The position confidence of the target candidate box is divided by the total confidence sum, to obtain a positioning certainty probability value of the target candidate box;

[0027] The positioning certainty probability value of the plurality of candidate boxes is calculated according to the calculation method of the positioning certainty probability value of the target candidate box.

[0028] The embodiments of the present disclosure process the position confidence of the plurality of candidate boxes by using the normalization processing method, so as to delete the redundant candidate boxes of the target detection object, and further achieve accurate positioning of the target detection object.

[0029] In some optional embodiments, the position confidence of the plurality of candidate boxes in the target detection region is suppressed, to obtain an optimal candidate box of the target detection object, including:

[0030] The position confidence of the plurality of candidate boxes in the target detection region is sequentially sorted in descending order, to obtain a candidate box corresponding to a maximum position confidence;

[0031] The candidate box corresponding to the maximum position confidence is taken as the optimal candidate box of the target detection object.

[0032] The embodiments of the present disclosure achieve accurate removal of the interference candidate box by using the above method, so as to obtain the optimal candidate box of the target detection object, and finally achieve the purpose of accurate positioning of the target detection object.

[0033] According to a second aspect, the present disclosure provides a three-dimensional object detection candidate box optimization device, the device comprising:

[0034] a confidence prediction module configured to predict position confidence of a plurality of candidate boxes of a target detection object based on three-dimensional features of the target detection object using a target neural network, wherein the target neural network is trained in advance;

[0035] a confidence processing module configured to process the plurality of candidate boxes of the target detection object in sequence through a segmentation processing mode and a normalization processing mode based on the position confidence of the plurality of candidate boxes of the target detection object, to obtain positioning certainty probability values of the plurality of candidate boxes;

[0036] a confidence determination module configured to obtain a target detection region of the target detection object and obtain position confidence of the plurality of candidate boxes in the target detection region based on the positioning certainty probability values of the plurality of candidate boxes;

[0037] an optimal box determination module configured to suppress the position confidence of the plurality of candidate boxes in the target detection region to obtain an optimal candidate box of the target detection object.

[0038] In a third aspect, the present disclosure provides a computer device, comprising a memory and a processor, the memory and the processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the three-dimensional object detection candidate box optimization method of the first aspect or any of the corresponding embodiments.

[0039] In a fourth aspect, the present disclosure provides a computer readable storage medium, the computer readable storage medium stores computer instructions, and the computer instructions are used to make a computer execute the three-dimensional object detection candidate box optimization method of the first aspect or any of the corresponding embodiments.

[0040] In a fifth aspect, the present disclosure provides a computer program product, comprising computer instructions, and the computer instructions are used to make a computer execute the three-dimensional object detection candidate box optimization method of the first aspect or any of the corresponding embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0042] Figure 1is a flowchart of a candidate box optimization method for three-dimensional detection objects according to an embodiment of the present application;

[0043] Figure 2 is a schematic diagram of a plurality of candidate boxes for target detection objects before optimization according to an embodiment of the present application;

[0044] Figure 3 is a schematic diagram of a plurality of candidate boxes for target detection objects after optimization according to an embodiment of the present application;

[0045] Figure 4 is another flowchart of a candidate box optimization method for three-dimensional detection objects according to an embodiment of the present application;

[0046] Figure 5 is a structural block diagram of a candidate box optimization device for three-dimensional detection objects according to an embodiment of the present application;

[0047] Figure 6 is a hardware structure schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] To make the objectives, technical solutions and advantages of embodiments of the present application clearer, the technical solutions in embodiments of the present application will be described below in connection with the drawings in embodiments of the present application. Obviously, the described embodiments are some but not all of embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0049] According to an embodiment of the present application, a candidate box optimization method for three-dimensional detection objects is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0050] In this embodiment, a candidate box optimization method for three-dimensional detection objects is provided, which can be used in computer devices such as mobile phones, tablet computers, desktop computers, portable notebooks, servers, etc. The candidate box optimization method for three-dimensional detection objects in the present disclosure can be specifically applied to target object detection in the scene of autonomous driving or robot or unmanned aerial vehicle navigation, helping the detection system to perceive and understand the surrounding environment in real time, and making safe and accurate decision tasks, such as making multi-target tracking, vehicle trajectory decision tasks, etc. Figure 1 is a flowchart of a candidate box optimization method for three-dimensional detection objects according to an embodiment of the present application, as shown in Figure 1 the flowchart includes the following steps:

[0051] Step S101, based on the three-dimensional features of the target detection object, using a target neural network to predict the position confidence of multiple candidate boxes of the target detection object, wherein the target neural network is trained in advance.

[0052] Specifically, the target detection object can be any one of the multiple detection objects in the current detection environment. For example, the embodiment of the present disclosure is applied in the automatic driving scene, and the target detection object can be a road sign, a bridge or a traffic signal lamp in the vehicle driving road. Through the three-dimensional features of the target detection object, the three-dimensional structure of the target detection object can be known, and then the spatial position, nature and dynamic change of the target detection object can be known. The perception technology of three-dimensional features is not only widely used in the scene of automatic driving or robot or unmanned aerial vehicle navigation, but also plays an irreplaceable role in the development of immersive experience such as augmented reality (AR) and virtual reality (VR), and in many fields such as medical treatment, construction and entertainment.

[0053] In some optional embodiments, the step S101 of obtaining the three-dimensional features of the target detection object comprises:

[0054] Step a1, obtaining grid-shaped sampling points from the target candidate box of the target detection object.

[0055] Specifically, the target candidate box is any one of the multiple candidate boxes, and the shape of any candidate box can be a rectangle. For example, grid-shaped sampling points are taken in the rectangular target candidate box, such as a 7x3x3 grid.

[0056] Step a2, projecting the grid-shaped sampling points to a two-dimensional space by using the camera internal and external parameters to obtain two-dimensional image data of the target detection object.

[0057] Specifically, the sampling points in the target candidate box are projected to a two-dimensional camera imaging plane by using the camera internal and external parameters to obtain two-dimensional image data of the target detection object.

[0058] Step a3, converting the two-dimensional image data of the target detection object into three-dimensional image data.

[0059] Step a4, extracting the three-dimensional features of the target detection object from the three-dimensional image data.

[0060] In a specific example, the step S101 of predicting the position confidence of multiple candidate boxes of the target detection object based on the three-dimensional features of the target detection object by using a target neural network, wherein the target neural network is trained in advance, comprises:

[0061] The three-dimensional features of the target detection object are input into the detection module of the target neural network for prediction to obtain the position confidence of multiple candidate boxes of the target detection object output by the target neural network.

[0062] Specifically, the target neural network is a general neural network with a detection head, the target neural network is used to model the position confidence of the target detection object, and the target neural network is pre-trained based on three-dimensional features in order to further locate the current position of the target detection object. The detection module is a network of the detection head in the target neural network, and the detection module is used to predict the position confidence of multiple candidate boxes of the target detection object, and the position confidence is used to represent the related information of the position of the target detection object in the image. The detection module of the target neural network is used to predict the position confidence of multiple candidate boxes of the target detection object, which can improve the robustness of the prediction result.

[0063] In step S102, based on the position confidence of the multiple candidate boxes of the target detection object, the multiple candidate boxes of the target detection object are processed in sequence through a segmentation processing mode and a normalization processing mode, and the positioning certainty probability value of the multiple candidate boxes is obtained.

[0064] Specifically, in the three-dimensional target detection process, the three-dimensional detection object is easily affected by occlusion or sensor noise, and therefore, too many redundant candidate boxes are generated, which finally easily leads to inaccurate detection results of the three-dimensional detection object. Therefore, it is necessary to further remove the redundant candidate boxes of the target detection object to ensure the accuracy of the positioning of the target detection object.

[0065] Further, the positioning certainty probability value represents the determination degree of the existence of voxel data of the target detection object at the position of the target candidate box, and it is an index for measuring the quality of the positioning information of the target detection object. Generally, the greater the positioning certainty probability value, the greater the reliability of the positioning information of the target detection object. The segmentation processing mode and the normalization processing mode are both used to delete the redundant candidate boxes of the target detection object, and thus realize accurate positioning of the target detection object.

[0066] In some optional embodiments, the multiple candidate boxes of the target detection object are optimized through the segmentation processing mode, which comprises:

[0067] In step c1, based on the position confidence of the multiple candidate boxes of the target detection object, a preset segmentation algorithm is used to perform first segmentation on the multiple candidate boxes of the target detection object which are not connected to each other.

[0068] Specifically, for example, a bird's eye view is generated based on the position confidence of the multiple candidate boxes of the target detection object, the bird's eye view represents the intensity distribution diagram of the position confidence of the multiple candidate boxes in the three-dimensional space, and a segmentation algorithm such as a connected domain segmentation algorithm is applied to separate the confidence distribution regions of the multiple candidate boxes which are not connected to each other.

[0069] Step c2, obtaining the mutually connected target detection objects from the first segmentation result of the plurality of candidate boxes, and performing second segmentation on the mutually connected target detection objects by using a neighborhood maximum suppression method.

[0070] Specifically, for example, as shown in Figure 2 If the regions where the plurality of candidate boxes of blue color and the plurality of candidate boxes of purple color are mutually connected, the blue region and the purple region are segmented by using the neighborhood maximum suppression method. The maximum suppression method generally refers to a non-maximum suppression (NMS) algorithm. The working principle of the NMS algorithm is based on the judgment of local maximum value. For any candidate box, the NMS algorithm calculates the overlap degree (usually using the intersection over union (IOU) as the measurement standard) between the candidate box and other candidate boxes. If the overlap degree between a certain candidate box and other candidate boxes exceeds a set threshold, and the position confidence of the certain candidate box is not the highest, the certain candidate box will be suppressed (i.e. filtered out), and those candidate boxes with the highest local position confidence and the overlap degree not exceeding the threshold are retained.

[0071] In some optional embodiments, the plurality of candidate boxes of the target detection object are processed by a normalization processing method to obtain positioning certainty probability values of the plurality of candidate boxes, including:

[0072] Step d1, calculating a total confidence sum of the target detection object based on the position confidence of the plurality of candidate boxes of the target detection object.

[0073] Specifically, the position confidence of the plurality of candidate boxes of the target detection object is 0.6, 0.5, 0.4, 0.3, 0.2, 0.1 and 0.9 respectively, and the confidence sum of the plurality of candidate boxes is 3.

[0074] Step d2, calculating the positioning certainty probability value of the target candidate box by dividing the position confidence of the target candidate box by the total confidence sum.

[0075] Specifically, the target candidate box is any candidate box of the plurality of candidate boxes, and the positioning certainty probability value of any target candidate box is 0.6 / 3=0.2, 0.5 / 3≈0.17, 0.4 / 3≈0.13, 0.3 / 3=0.1, 0.2 / 3≈0.07, 0.1 / 3≈0.03 and 0.9 / 3=0.3 respectively.

[0076] Step d3, calculating the positioning certainty probability values of the plurality of candidate boxes according to the calculation method of the positioning certainty probability value of the target candidate box.

[0077] Specifically, according to the above calculation method, the positioning determination degree probability values of the plurality of candidate boxes are 0.2, 0.17, 0.13, 0.1, 0.07, 0.03, and 0.3, respectively.

[0078] In step S103, a target detection region of the target detection object is obtained based on the positioning determination degree probability values of the plurality of candidate boxes, and a position confidence of the plurality of candidate boxes in the target detection region is obtained.

[0079] Specifically, for example, in the case of the red candidate boxes, the target detection region is the region where the plurality of red candidate boxes are located, and the position confidence of the plurality of red candidate boxes is further obtained. Figure 2

[0080] In some optional embodiments, the step S103 of obtaining the target detection region of the target detection object based on the positioning determination degree probability values of the plurality of candidate boxes comprises:

[0081] In step e1, it is determined whether the positioning determination degree probability value of the target candidate box of the target detection object is greater than a preset threshold value, wherein the target candidate box is any candidate box in the plurality of candidate boxes.

[0082] Specifically, also for the case of the red candidate boxes, it is determined whether the positioning determination degree probability value of the target candidate box (any red candidate box) is greater than a preset threshold value. Figure 2 The preset threshold value is a reference value preselected, and the preset threshold value can be flexibly set in combination with an actual application scenario. The preset threshold value can be set in combination with some parameters with smaller positioning determination degree probability values, so as to filter out the candidate boxes with smaller positioning determination degree probability values through the preset threshold value.

[0083] In step e2, if the positioning determination degree probability value of the target candidate box of the target detection object is greater than the preset threshold value, it is determined that there is a voxel in the region where the target candidate box is located.

[0084] Specifically, if the positioning determination degree probability value of the target candidate box (any candidate box) of the target detection object is greater than the preset threshold value, it is determined that there is a voxel in the region where the target candidate box (any candidate box) is located, and otherwise, there is no voxel in the region where the target candidate box (any candidate box) is located.

[0085] In step e3, if the positioning determination degree probability value of the target candidate box of the target detection object is not greater than the preset threshold value, the target candidate box is deleted.

[0086] Specifically, for example, in the case of the red candidate boxes, if the positioning determination degree probability value of the target candidate box of the target detection object is not greater than the preset threshold value, the corresponding candidate box is deleted. Figure 2

[0087] ​​The embodiment of the present disclosure further determines whether the positioning certainty probability value of the target candidate box of the target detection object is greater than a preset threshold based on the positioning certainty probability values of the plurality of candidate boxes obtained based on the normalization processing result, so as to screen the candidate box with a better positioning certainty probability value, thereby facilitating accurate removal of irrelevant redundant candidate boxes.

[0088] Step e4, taking the entire region with voxels as the target detection region of the target detection object.

[0089] Specifically, the entire region with voxels is taken as the target detection region of the target detection object based on the determination result. Figure 2 For example, in the case of , if the regions of the plurality of red candidate boxes all have voxels, the regions of the plurality of red candidate boxes are taken as the target detection region of the same target detection object.

[0090] Step S104, suppressing the position confidence of the plurality of candidate boxes in the target detection region to obtain the optimal candidate box of the target detection object.

[0091] In some optional embodiments, the step S104 of suppressing the position confidence of the plurality of candidate boxes in the target detection region to obtain the optimal candidate box of the target detection object includes:

[0092] Step d1, sequentially sorting the position confidence of the plurality of candidate boxes in the target detection region in descending order to obtain the candidate box corresponding to the maximum position confidence.

[0093] Step d2, taking the candidate box corresponding to the maximum position confidence as the optimal candidate box of the target detection object.

[0094] Figure 2 Specifically, for example, in the case of Figure 2 , for example, the position confidence of the plurality of red candidate boxes sequentially in descending order is 0.8, 0.75, 0.63, 0.61, and 0.55, and the candidate box corresponding to the maximum confidence 0.8 is selected as the optimal candidate box. Figure 2 In the case of Figure 3 , the plurality of candidate boxes in the same color set represent the same target detection object, and the plurality of candidate boxes in the same color set in are all optimized according to the steps S101-S104, and further obtain the optimization result as shown in

[0095] .The three-dimensional object candidate box optimization method in the embodiments of the present disclosure is based on the three-dimensional features of the target detection object, uses a target neural network to predict the position confidence of multiple candidate boxes of the target detection object, wherein the target neural network is trained in advance; based on the position confidence of the multiple candidate boxes of the target detection object, the multiple candidate boxes of the target detection object are processed in turn through a segmentation processing mode and a normalization processing mode to obtain the positioning certainty probability value of the multiple candidate boxes; based on the positioning certainty probability value of the multiple candidate boxes, the target detection region of the target detection object is obtained, and the position confidence of the multiple candidate boxes in the target detection region is obtained; the position confidence of the multiple candidate boxes in the target detection region is suppressed to obtain the optimal candidate box of the target detection object, so that the embodiments of the present disclosure not only help to accurately remove the redundant candidate boxes of the target detection object, but also help to accurately position the target detection object.

[0096] In the embodiments, a three-dimensional object candidate box optimization method is provided, which can be used in computer devices such as mobile phones, tablet computers, desktop computers, portable notebooks, servers, etc. The three-dimensional object candidate box optimization method in the embodiments of the present disclosure can be specifically applied to target object detection in the scene of automatic driving or robot or unmanned aerial vehicle navigation, helping the detection system to perceive and understand the surrounding environment in real time, and making safe and accurate decision tasks, such as multi-target tracking, vehicle running trajectory decision tasks, etc. Figure 4 The three-dimensional object candidate box optimization method according to the embodiments of the present disclosure is shown in the flowchart as shown in Figure 4 The flowchart includes the following steps:

[0097] In step S101, based on the three-dimensional features of the target detection object, the position confidence of multiple candidate boxes of the target detection object is predicted using a target neural network, wherein the target neural network is trained in advance.

[0098] In step S102, based on the position confidence of the multiple candidate boxes of the target detection object, the multiple candidate boxes of the target detection object are processed in turn through a segmentation processing mode and a normalization processing mode to obtain the positioning certainty probability value of the multiple candidate boxes.

[0099] In step S103, based on the positioning certainty probability value of the multiple candidate boxes, the target detection region of the target detection object is obtained, and the position confidence of the multiple candidate boxes in the target detection region is obtained.

[0100] In step S104, the position confidence of the multiple candidate boxes in the target detection region is suppressed to obtain the optimal candidate box of the target detection object.

[0101] Specifically, the above steps S101-S104 have been described in the above process, and will not be repeated here.

[0102] In step S105, the target detection object is positioned in the target detection region based on the optimal candidate box of the target detection object.

[0103] The embodiments of the present disclosure position the target detection object in the target detection region based on the optimal candidate box of the target detection object, which is beneficial to accurately track the target detection object in different driving scenes.

[0104] In the present embodiment, a candidate box optimization device for a three-dimensional detection object is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and contemplated.

[0105] The present embodiment provides a candidate box optimization device for a three-dimensional detection object, as shown in Figure 5 The device comprises:

[0106] A confidence prediction module 501 is configured to predict position confidence of a plurality of candidate boxes of a target detection object based on three-dimensional features of the target detection object by using a target neural network, wherein the target neural network is trained in advance.

[0107] A confidence processing module 502 is configured to process the plurality of candidate boxes of the target detection object by using a segmentation processing mode and a normalization processing mode in sequence based on the position confidence of the plurality of candidate boxes, to obtain a positioning certainty probability value of the plurality of candidate boxes.

[0108] A confidence determination module 503 is configured to obtain a target detection region of the target detection object and obtain position confidence of the plurality of candidate boxes in the target detection region based on the positioning certainty probability value of the plurality of candidate boxes.

[0109] An optimal box determination module 504 is configured to suppress the position confidence of the plurality of candidate boxes in the target detection region to obtain an optimal candidate box of the target detection object.

[0110] In some optional embodiments, the confidence determination module 503 comprises:

[0111] A candidate box judgment sub-module is configured to judge whether the positioning certainty probability value of a target candidate box of the target detection object is greater than a preset threshold, wherein the target candidate box is any candidate box in the plurality of candidate boxes.

[0112] A first determination sub-module is configured to determine that a region where the target candidate box is located exists voxels if the positioning certainty probability value of the target candidate box of the target detection object is greater than the preset threshold.

[0113] a candidate box deletion submodule, configured to delete a target candidate box of the target detection object if a positioning certainty probability value of the target candidate box is not greater than a preset threshold value;

[0114] a second determination submodule, configured to take all regions of the existing voxels as a target detection region of the target detection object.

[0115] In some optional embodiments, the three-dimensional detection object candidate box optimization apparatus in the embodiments of the present disclosure is configured to Figure 5 In some optional embodiments, the three-dimensional detection object candidate box optimization apparatus in the embodiments of the present disclosure is configured to

[0116] a target positioning module 505, configured to position the target detection object in the target detection region based on the optimal candidate box of the target detection object.

[0117] In some optional embodiments, the confidence processing module 502 comprises:

[0118] a first segmentation submodule, configured to perform first segmentation on the multiple candidate boxes of the target detection object that are not connected to each other by using a preset segmentation algorithm based on the position confidence of the multiple candidate boxes of the target detection object;

[0119] a second segmentation submodule, configured to obtain the target detection object that is connected to each other from the first segmentation result of the multiple candidate boxes, and perform second segmentation on the target detection object that is connected to each other by using a neighborhood maximum suppression manner.

[0120] In some optional embodiments, the confidence processing module 502 comprises:

[0121] a first calculation submodule, configured to calculate a total confidence sum of the target detection object based on the position confidence of the multiple candidate boxes of the target detection object;

[0122] a second calculation submodule, configured to calculate the positioning certainty probability value of the target candidate box by dividing the position confidence of the target candidate box by the total confidence sum;

[0123] a third calculation submodule, configured to calculate the positioning certainty probability value of the multiple candidate boxes in the manner of calculating the positioning certainty probability value of the target candidate box.

[0124] In some optional embodiments, the optimal box determination module 504 comprises:

[0125] a confidence sorting submodule, configured to sort the position confidence of the multiple candidate boxes in the target detection region in descending order to obtain a candidate box corresponding to a maximum position confidence;

[0126] a candidate box determination submodule, configured to take the candidate box corresponding to the maximum position confidence as the optimal candidate box of the target detection object.

[0127] Further function description of each module and unit is the same as the corresponding embodiment described above, which will not be repeated here.

[0128] The candidate box optimization apparatus for three-dimensional object detection in the embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories executing one or more software or fixed programs, and / or other devices that can provide the above functions.

[0129] The embodiment of the present application also provides a computer device with the candidate box optimization apparatus for three-dimensional object detection.

[0130] Please refer to Figure 6 , Figure 6 is a structural schematic diagram of a computer device provided by an optional embodiment of the present application, as shown in Figure 6 the computer device includes one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Various components are communicatively connected to each other by different buses, and can be installed on a common mainboard or in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in the memory or on the memory to display graphical information on a GUI of an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memories, if necessary. Similarly, multiple computer devices can be connected, each providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 In the embodiment, the processor 10 is taken as an example.

[0131] The processor 10 can be a central processor, a network processor, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic gate array, a generic array logic, or any combination thereof.

[0132] The memory 20 stores instructions executable by the at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0133] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required for at least one function, etc. The data storage area can store data created by the computer device, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-transitory memory such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid state memory device. In some alternative embodiments, the memory 20 can optionally include memory that is remotely located with respect to the processor 10, and which can be connected to the computer device through a network. Examples of such networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communications network, and combinations thereof.

[0134] The memory 20 can include a volatile memory, such as a random access memory, and / or can include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid state memory device. The memory 20 can also include an array of multi-state flash memory cells, which can be used to store data and / or instructions in multiple states.

[0135] The computer device also includes a communications interface 30 for communicating with other devices or communication networks.

[0136] The embodiments of the present application also provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or implemented as computer code to be originally stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and stored in a local storage medium, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that the computer, processor, microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the method shown in the above embodiments.

[0137] Part of the present application can be applied as a computer program product, for example, computer program instructions, when executed by a computer, through the operation of the computer, can invoke or provide the method and / or technical solutions according to the present application. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source files, executable files, installation package files and the like, and accordingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0138] Although the embodiments of the present application are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method for optimizing bounding boxes of detected objects in three dimensions, the method comprising: The method comprises: Based on the three-dimensional features of the target detection object, the position confidence of the multiple candidate boxes of the target detection object is predicted by using a target neural network, wherein the target neural network is trained in advance, and the position confidence is used to represent the relevant information of the position of the target detection object in the image; Based on the position confidence of the multiple candidate boxes of the target detection object, the multiple candidate boxes of the target detection object are processed in sequence by using a segmentation processing mode and a normalization processing mode to obtain the positioning certainty probability value of the multiple candidate boxes; wherein the processing of the multiple candidate boxes of the target detection object by using the segmentation processing mode comprises: Based on the position confidence of the multiple candidate boxes of the target detection object, the multiple candidate boxes of the target detection object which are not connected to each other are subjected to first segmentation by using a preset segmentation algorithm; From the first segmentation result of the multiple candidate boxes, the target detection objects which are connected to each other are obtained, and the target detection objects which are connected to each other are subjected to second segmentation by using a neighborhood maximum suppression mode; The multiple candidate boxes of the target detection object are processed by using the normalization processing mode to obtain the positioning certainty probability value of the multiple candidate boxes, comprising: Based on the position confidence of the multiple candidate boxes of the target detection object, the total confidence sum of the target detection object is calculated; The position confidence of the target candidate box is divided by the total confidence sum to obtain the positioning certainty probability value of the target candidate box; The positioning certainty probability value of the multiple candidate boxes is calculated in the manner of the positioning certainty probability value of the target candidate box; Based on the positioning certainty probability value of the multiple candidate boxes, the target detection region of the target detection object is obtained, and the position confidence of the multiple candidate boxes in the target detection region is obtained; The position confidence of the multiple candidate boxes in the target detection region is suppressed to obtain the optimal candidate box of the target detection object.

2. The method of claim 1, wherein, Based on the positioning certainty probability value of the multiple candidate boxes, the target detection region of the target detection object is obtained, comprising: It is judged whether the positioning certainty probability value of the target candidate box of the target detection object is greater than a preset threshold, wherein the target candidate box is any candidate box in the multiple candidate boxes; If the positioning certainty probability value of the target candidate box of the target detection object is greater than the preset threshold, it is determined that there is a voxel in the region of the target candidate box; If the positioning certainty probability value of the target candidate box of the target detection object is not greater than the preset threshold, the target candidate box is deleted; All regions with voxels are taken as the target detection region of the target detection object.

3. The method of claim 1, wherein, Further comprising: Based on the optimal candidate box of the target detection object, the target detection object in the target detection region is located.

4. The method of claim 1, wherein, The positioning certainty probability value of the multiple candidate boxes in the target detection region is suppressed to obtain the optimal candidate box of the target detection object, comprising: The position confidence of the multiple candidate boxes in the target detection region is sequentially sorted in descending order to obtain the candidate box corresponding to the maximum position confidence; The candidate box corresponding to the maximum position confidence is taken as the optimal candidate box of the target detection object.

5. An apparatus for optimizing candidate boxes of a three-dimensional detected object, characterized in that, The device comprises: The confidence prediction module is configured to predict position confidence of the multiple candidate boxes of the target detection object based on three-dimensional features of the target detection object by using a target neural network, wherein the target neural network is trained in advance, and the position confidence is used to represent relevant information of a position of the target detection object in an image. The confidence processing module is configured to process the multiple candidate boxes of the target detection object by using a segmentation processing mode and a normalization processing mode in sequence based on the position confidence of the multiple candidate boxes of the target detection object, to obtain a positioning certainty probability value of the multiple candidate boxes. The first segmentation submodule is configured to perform first segmentation on the multiple candidate boxes of the target detection object that are not connected to each other by using a preset segmentation algorithm based on the position confidence of the multiple candidate boxes of the target detection object. The second segmentation submodule is configured to obtain the target detection object that is connected to each other from the first segmentation result of the multiple candidate boxes, and perform second segmentation on the target detection object that is connected to each other by using a neighborhood maximum suppression mode. The confidence processing module includes: The first calculation submodule is configured to calculate a total confidence sum of the target detection object based on the position confidence of the multiple candidate boxes of the target detection object. The second calculation submodule is configured to calculate the positioning certainty probability value of the target candidate box by dividing the position confidence of the target candidate box by the total confidence sum. The third calculation submodule is configured to calculate the positioning certainty probability value of the multiple candidate boxes in the manner of calculating the positioning certainty probability value of the target candidate box. The confidence determination module is configured to obtain a target detection region of the target detection object based on the positioning certainty probability value of the multiple candidate boxes, and obtain the position confidence of the multiple candidate boxes in the target detection region. The optimal box determination module is configured to suppress the position confidence of the multiple candidate boxes in the target detection region, to obtain an optimal candidate box of the target detection object.

6. A computer device, comprising: The memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the three-dimensional detection object candidate box optimization method in any one of claims 1 to 4. The computer readable storage medium stores computer instructions, and the computer instructions are used to make a computer execute the three-dimensional detection object candidate box optimization method in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer instructions are used to make a computer execute the three-dimensional detection object candidate box optimization method in any one of claims 1 to 4.

8. A computer program product, characterised in that, ​

Images