Three-dimensional target detection model optimization method and related device

Abstract

The invention belongs to the field of target detection, and discloses a three-dimensional target detection model optimization method and a related device, and the method comprises the steps: obtaining training data, carrying out the N times of training of a three-dimensional target detection model through the training data, and obtaining N pre-trained three-dimensional target detection models; obtaining a positioning confidence coefficient variance of each detection frame of each pre-trained three-dimensional target detection model, and obtaining an anomaly detection frame according to a preset positioning confidence coefficient variance threshold; and placing a truth value box corresponding to the anomaly detection box in a plurality of target point cloud frames of training data under the condition that the geometric position is not changed, and when the three-dimensional target detection model shows consistent anomaly detection behaviors under the plurality of target point cloud frames, performing data augmentation on the training data and then training the three-dimensional target detection model. The detection precision and efficiency of the three-dimensional target detection model are remarkably improved, and the model is endowed with higher generalization ability, so that the model can better adapt to complexity and uncertainty in various practical application scenes.

Description

Technical Field

[0001] The invention belongs to the field of target detection and relates to a three-dimensional target detection model optimization method and related devices. Background Art

[0002] 3D object detection technology is a core component of autonomous intelligent unmanned systems, which can provide intelligent agents with information such as the distance, direction, and category of surrounding obstacles. Algorithm developers can identify and locate problems that affect detection performance in 3D object detection models and datasets through in-depth analysis of detection results, and based on this, they can derive strategies for tuning 3D object detection models, thereby improving 3D object detection models.

[0003] However, existing tools for evaluating 3D object detection results often only provide quantitative indicators at the dataset level such as Average Precision (AP), and provide visualization results of 3D boxes such as point clouds, detection boxes, and true value boxes, but do not intuitively display specific details such as the Intersection over Union (IoU) between the detection box and the true value box, 3D box category information, detection box confidence, and detection results (i.e., true positive, false positive, and false negative). This limits the ability of algorithm developers to fully analyze and deeply explore the detection results of 3D object detection models, reduces the optimization effect of 3D object detection models, and thus results in low accuracy and efficiency when using 3D object detection models for 3D object detection. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide a three-dimensional target detection model optimization method and related devices.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a three-dimensional target detection model optimization method, comprising: obtaining training data, and training the three-dimensional target detection model N times using the training data to obtain N pre-trained three-dimensional target detection models; obtaining the positioning confidence variance of each detection frame of each pre-trained three-dimensional target detection model, and obtaining an anomaly detection frame based on a preset positioning confidence variance threshold; placing a true value frame corresponding to the anomaly detection frame in several target point cloud frames of the training data while keeping the geometric position unchanged, and when the three-dimensional target detection model exhibits consistent anomaly detection behavior in the several target point cloud frames, performing data augmentation on the training data and then training the three-dimensional target detection model.

[0007] Optionally, it also includes: obtaining a changing trend of the intersection ratio of the anomaly detection frame and the true value frame by adjusting the length, width, height and geometric center point of the prior box of the three-dimensional target detection model, the distribution of laser points in the prior box or the orientation of the prior box, and adjusting the length, width, height and geometric center point of the prior box of the three-dimensional target detection model, the distribution of laser points in the prior box or the orientation of the prior box according to the changing trend.

[0008] Optionally, before training the three-dimensional target detection model N times using the training data, the method further includes: cleaning the training data based on preset cleaning rules; wherein the preset cleaning rules include: obtaining the number of laser points in the true value frame corresponding to each label of each point cloud frame in the training data, and removing the label corresponding to the true value frame whose number of laser points in the true value frame is less than a preset laser point threshold.

[0009] Optionally, the obtaining of the positioning reliability variance of each detection frame of each pre-trained 3D target detection model, and obtaining the abnormal detection frame according to a preset positioning reliability variance threshold includes: constructing a 3D frame similarity matrix of each pre-trained 3D target detection model based on the Euclidean distance between the geometric center points of the detection frame and the true value frame, and processing the 3D frame similarity matrix of each pre-trained 3D target detection model based on the Hungarian algorithm to obtain a one-to-one pairing relationship between the detection frame and the true value frame of each pre-trained 3D target detection model, and setting the detection frame paired with the i-th true value frame as the i-th detection frame; and setting the positioning reliability of the detection frame whose similarity with the paired true value frame is lower than the preset similarity threshold to 0; obtaining the positioning reliability variance v of the i-th detection frame by the following formula i :

[0010]

[0011] in, is the positioning confidence of the i-th detection frame of the n-th pre-trained 3D target detection model; the detection frame whose positioning confidence variance is less than the preset positioning confidence variance threshold is an abnormal detection frame.

[0012] Optionally, it also includes: visual display of the three-dimensional intersection ratio, two-dimensional top view intersection ratio, two-dimensional image intersection ratio, positioning reliability of the detection frame, the number of laser points of the detection frame and the number of laser points of the true value frame of each pre-trained three-dimensional target detection model.

[0013] Optionally, it also includes: visually displaying the ray OA′ of the three-dimensional target detection model 3D and ray OB′ 3D ; Among them, ray OA′ 3D and ray OB′ 3D Point A′ 3D and point B′ 3D The ray to the camera's optical center O; For LiDAR To the camera system The coordinate system transformation matrix, M is the camera's intrinsic parameter matrix, and λ is a preset constant; A 2D and B 2D A is the top left corner of the 3D detection frame of the 3D target detection model. 3D and the front upper right corner point B 3D The imaging point on the camera image plane; when the ray OA ′ 3D and ray OB3 ′ D Departure point A 3D and point B 3D When adjusting the external parameter matrix of the laser radar and camera of the 3D target detection model, the ray OA ′ 3D and ray OB3 ′ D Pass point A 3D and point B 3D .

[0014] Optionally, the data augmentation on the training data includes: performing one or more of the following data augmentation operations on the training data: detection instance augmentation, point cloud scaling, point cloud translation, and point cloud rotation.

[0015] According to a second aspect of the present invention, a three-dimensional target detection model optimization system is provided, comprising: a data acquisition module, used to acquire training data, and train the three-dimensional target detection model N times using the training data to obtain N pre-trained three-dimensional target detection models; an anomaly determination module, used to acquire the positioning reliability variance of each detection frame of each pre-trained three-dimensional target detection model, and obtain an anomaly detection frame according to a preset positioning reliability variance threshold; and a model optimization module, used to place a truth frame corresponding to the anomaly detection frame in a number of target point cloud frames of the training data while keeping the geometric position unchanged, and when the three-dimensional target detection model exhibits consistent anomaly detection behavior in the several target point cloud frames, the three-dimensional target detection model is trained after data augmentation is performed on the training data.

[0016] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned three-dimensional target detection model optimization method when executing the computer program.

[0017] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned three-dimensional target detection model optimization method are implemented.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The three-dimensional target detection model optimization method of the present invention first performs multiple rounds of training on the three-dimensional target detection model to generate a series of pre-trained models, and builds a comprehensive model performance evaluation framework based on this. Subsequently, the positioning reliability variance analysis is introduced to measure the positioning accuracy fluctuation of each pre-trained model detection frame in detail, and according to the preset positioning reliability variance threshold, the abnormal detection frame is accurately screened out to reveal the potential weaknesses of the model under specific circumstances. Then, the true value frames corresponding to these abnormal detection frames are embedded into multiple representative target point cloud frames while keeping their original geometric positions unchanged, so as to simulate the complex scenes of the real world. By observing the performance of the model under these specific conditions, especially when the model shows consistent abnormal detection behavior under multiple target point cloud frames, it is shown that the model may have inherent defects when processing data of this type of specific structure or pattern, and then a data augmentation strategy is adopted to optimize the training data set, and by increasing the diversity and complexity of the training data, the model is helped to learn a wider and deeper feature representation, thereby effectively overcoming the limitations of its detection performance. It not only significantly improves the detection accuracy and efficiency of the 3D object detection model, but also gives the model stronger generalization capabilities, enabling it to better adapt to the complexity and uncertainty in various practical application scenarios. It not only deeply analyzes the potential problems of the 3D object detection model in the detection process, but also achieves precise tuning of the model through targeted means such as data augmentation, opening up a new path for improving the overall performance and reliability of 3D object detection technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of a three-dimensional object detection model optimization method according to an embodiment of the present invention.

[0021] Figure 2 A two-dimensional perspective view of a camera according to an embodiment of the present invention.

[0022] Figure 3 This is a three-dimensional top view of the point cloud according to an embodiment of the present invention.

[0023] Figure 4 This is a structural block diagram of a three-dimensional object detection model optimization system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0027] See also Figure 1 In one embodiment of the present invention, a three-dimensional target detection model optimization method is provided, which can effectively detect the problems that affect the detection performance in the model and data set, and obtain a method for model tuning based on this, thereby improving the three-dimensional target detection model and enhancing the accuracy and efficiency of three-dimensional target detection.

[0028] Specifically, the three-dimensional object detection model optimization method of the present invention includes the following steps:

[0029] S1: Obtain training data, and train the 3D object detection model N times using the training data to obtain N pre-trained 3D object detection models;

[0030] S2: Obtain the positioning confidence variance of each detection frame of each pre-trained 3D object detection model, and obtain an abnormal detection frame according to a preset positioning confidence variance threshold;

[0031] S3: The true value box corresponding to the anomaly detection box is placed in several target point cloud frames of the training data without changing its geometric position. When the three-dimensional object detection model exhibits consistent anomaly detection behavior in several target point cloud frames, the three-dimensional object detection model is trained after data augmentation of the training data.

[0032] The three-dimensional target detection model optimization method of the present invention first performs multiple rounds of training on the three-dimensional target detection model to generate a series of pre-trained models, and builds a comprehensive model performance evaluation framework based on this. Subsequently, the positioning reliability variance analysis is introduced to measure the positioning accuracy fluctuation of each pre-trained model detection frame in detail, and according to the preset positioning reliability variance threshold, the abnormal detection frame is accurately screened out to reveal the potential weaknesses of the model under specific circumstances. Then, the true value frames corresponding to these abnormal detection frames are embedded into multiple representative target point cloud frames while keeping their original geometric positions unchanged, so as to simulate the complex scenes of the real world. By observing the performance of the model under these specific conditions, especially when the model shows consistent abnormal detection behavior under multiple target point cloud frames, it is shown that the model may have inherent defects when processing data of this type of specific structure or pattern, and then a data augmentation strategy is adopted to optimize the training data set, and by increasing the diversity and complexity of the training data, the model is helped to learn a wider and deeper feature representation, thereby effectively overcoming the limitations of its detection performance. It not only significantly improves the detection accuracy and efficiency of the 3D object detection model, but also gives the model stronger generalization capabilities, enabling it to better adapt to the complexity and uncertainty in various practical application scenarios. It not only deeply analyzes the potential problems of the 3D object detection model in the detection process, but also achieves precise tuning of the model through targeted means such as data augmentation, opening up a new path for improving the overall performance and reliability of 3D object detection technology.

[0033] In a possible implementation, the three-dimensional target detection model optimization method also includes: obtaining a changing trend of the intersection ratio of the anomaly detection frame and the true value frame by adjusting the length, width, height and geometric center point of the prior box of the three-dimensional target detection model, the distribution of laser points in the prior box or the orientation of the prior box, and adjusting the length, width, height and geometric center point of the prior box of the three-dimensional target detection model, the distribution of laser points in the prior box or the orientation of the prior box according to the changing trend.

[0034] Specifically, a strategy is proposed to optimize the model performance by fine-tuning the prior box parameters of the 3D object detection model. Specifically, this strategy involves fine-tuning the length, width, height, geometric center point, distribution of laser points in the prior box, and orientation of the prior box, aiming to observe how these adjustments affect the change trend of the intersection over union (IoU) between the anomaly detection box and the true value box, and then optimize the model based on the change trend. For example, if fine-tuning the length, width, and height can improve the intersection ratio, the length, width, and height parameters of the model prior box can be set to the parameters used for fine-tuning.

[0035] This strategy can accurately locate the model weaknesses. By adjusting the prior box parameters and observing the IoU change trend, the model's weaknesses in dealing with targets of specific shapes, sizes, or orientations can be more accurately identified. This refined analysis provides a clear direction for subsequent model optimization. According to the IoU change trend, the parameters of the prior box can be dynamically adjusted to make it closer to the characteristics of the real target. This dynamic optimization process not only improves the detection accuracy of the model, but also enhances the model's adaptability to targets of different shapes, sizes, and orientations. By continuously iteratively adjusting the prior box parameters and retraining the model, the model's deviation in dealing with specific types of targets can be gradually eliminated, thereby improving the model's generalization ability. This enables the model to perform more stable and reliable performance when facing unknown or complex scenarios. Compared with traditional model tuning methods, the method of optimizing model performance by adjusting the prior box parameters is more direct and efficient. It allows the detection performance of the model to be quickly improved by fine-tuning the prior box parameters without changing the overall structure of the model.

[0036] In a possible implementation, before training the three-dimensional target detection model N times using the training data, the method further includes: cleaning the training data based on a preset cleaning rule; wherein the preset cleaning rule includes: obtaining the number of laser points in the true value frame corresponding to each label of each point cloud frame in the training data, and removing the label corresponding to the true value frame whose number of laser points in the true value frame is less than a preset laser point threshold.

[0037] Specifically, the main purpose of data cleaning is to remove samples with high geometric uncertainty (i.e., difficult samples) from the training set of the database to ensure the training effect of the 3D object detection model.

[0038] In a possible implementation, the obtaining of the positioning reliability variance of each detection frame of each pre-trained three-dimensional object detection model, and obtaining the abnormal detection frame according to a preset positioning reliability variance threshold includes:

[0039] Based on the Euclidean distance between the geometric center points of the detection box and the true value box, a 3D box similarity matrix of each pre-trained 3D target detection model is constructed, and the 3D box similarity matrix of each pre-trained 3D target detection model is processed based on the Hungarian algorithm to obtain a one-to-one pairing relationship between the detection box and the true value box of each pre-trained 3D target detection model, and the detection box paired with the i-th true value box is set to the i-th detection box; and the position confidence of the detection box whose similarity with the paired true value box is lower than the preset similarity threshold is set to 0.

[0040] The position confidence variance v of the i-th detection box is obtained by the following formula: i :

[0041]

[0042] in, is the positioning confidence of the i-th detection box of the n-th pre-trained 3D object detection model.

[0043] A detection box whose positioning confidence variance is less than a preset positioning confidence variance threshold is an abnormal detection box.

[0044] Specifically, the anomaly detection frame can be divided into the following steps:

[0045] (1) When the model structure is consistent, use the system time as input to generate a random seed for random numbers to influence the input behavior of the model (such as using different point cloud data as input), the initialization behavior of the model weights, etc., so that N different models can be obtained through N trainings. (2) Construct a three-dimensional box similarity matrix. Use the Euclidean distance between the geometric center points of the detection box and the true value box to obtain the value of the matrix, that is, the similarity of the three-dimensional box. The smaller the value, the higher the overlap and the better the match between the detection box and the true value box. (3) Based on the Hungarian algorithm, process the three-dimensional box similarity matrix to obtain a one-to-one pairing relationship between the detection box and the true value box, mark the paired detection boxes, and give the i-th true value box its matching detection box is called the i-th detection box. If the similarity between a detection box and its paired true value box is low, the detection box is regarded as a Class A detection box. (4) Process the N groups of detection box results of N models and calculate the localization confidence variance v of the i-th detection box i The positioning confidence of the i-th detection box obtained by the n-th model is If the detection frame is a type A detection frame, then Set to 0. Finally, the detection box with a larger position confidence is considered as an abnormal detection box.

[0046] In a possible implementation, the three-dimensional target detection model optimization method also includes: visually displaying the three-dimensional intersection ratio, two-dimensional top view intersection ratio, two-dimensional image intersection ratio, positioning reliability of the detection frame, the number of laser points of the detection frame, and the number of laser points of the true value frame of each pre-trained three-dimensional target detection model.

[0047] Specifically, the sensor data such as point cloud in the database is input into the 3D object detection model to obtain the detection box information. The label information includes but is not limited to the length, width and height of the detected object, the coordinates and heading angle of the geometric center point in the sensor system, and the category information or category confidence. The detection process includes but is not limited to extracting 3D features from the point cloud using 3D sparse convolution, performing feature aggregation and pooling operations on the 3D features to obtain 2D features, and inputting the 2D features into the region candidate box network for regression and classification to obtain label information.

[0048] The model performance is then evaluated using the true value labels and predicted labels. Specifically, (1) Based on the geometric center point, heading angle, length, width, and height of the true value frame and the detection frame, the 3D intersection ratio, 2D top view intersection ratio, and 2D image intersection ratio between the frames are calculated, and the result of each detection instance (i.e., true positive, false positive, or false negative) is obtained based on the size of the intersection ratio. (2) The number of laser points in the i-th 3D frame is calculated, where the 3D frame includes the detection frame and the true value frame. According to the length l of the 3D frame i 、Width i , high h i , 3D frame in LiDAR system The heading angle θ i , geometric center point coordinates The first laser point under the laser radar system Based on the following formula group, the three-dimensional frame system can be calculated Laser point under Three-dimensional frame The laser point collection under and 3D frame system Laser points under

[0049]

[0050] in, Indicates that the coordinates are from the laser radar system To 3D frame The coordinate system transformation matrix; Represents the set of laser points in the lidar system.

[0051] Optionally, in this embodiment, the Ogre3D graphics rendering engine is used to visualize the 3D point cloud, 3D box, text and line. The interactive interface is designed based on the Python language and the DearPyGUI graphical function library, specifically including the sequence number of the key scene, visual display of the data of the corresponding scene, where the scene refers to the point cloud and image data collected at a certain moment, as well as the true value label and predicted data; through the interface controls, the display and hiding of the point cloud and bounding box are controlled; the forward and backward playback of the scene data and the reset of the viewing angle are controlled by keyboard shortcuts; when the detection box is clicked, the detection information of the detection box can be enlarged and displayed.

[0052] Optionally, the text information to be displayed on the interface can be adjusted through controls, including 3D intersection ratio, 2D top view intersection ratio, 2D image intersection ratio, positioning confidence, and the number of laser points in the 3D box. The point cloud of the true value box or the point cloud of the detection box can also be highlighted through controls.

[0053] In a possible implementation, the three-dimensional target detection model optimization method further includes: visually displaying the ray OA′ of the three-dimensional target detection model 3D and ray OB′ 3D ; Among them, ray OA′ 3D and ray OB′ 3D Point A′ 3D and point B′ 3D The ray to the camera's optical center O; For LiDAR To the camera system The coordinate system transformation matrix, M is the camera's intrinsic parameter matrix, and λ is a preset constant; A 2D and B 2D A is the top left corner of the 3D detection frame of the 3D target detection model. 3D and the front upper right corner point B 3D The imaging point on the camera image plane; when the ray OA′ 3D and ray OB′ 3D Departure point A 3D and point B 3D When adjusting the external parameter matrix of the laser radar and camera of the 3D target detection model, to ray OA′ 3D and ray OB′ 3D Pass point A 3D and point B 3D .

[0054] Specifically, 3D target detection is part of the robot's perception of the environment. During the perception process, the robot uses the onboard laser radar and onboard camera to collect environmental information around the body, and obtains two types of data, namely 3D point cloud and image. However, since the installation positions of the onboard laser radar and onboard camera on the body do not overlap, the two types of data are not synchronized in space. This data asynchrony will reduce the accuracy of 3D target detection based on image and point cloud fusion. To achieve spatial synchronization of the two types of data, the camera's intrinsic parameter matrix M and the laser radar system can be used. To the camera system The coordinate system transformation matrix To achieve spatial synchronization of data. Among them, the laser radar system is a three-dimensional coordinate system, the origin of which coincides with the geometric center of the laser radar, the x-axis points to the forward direction of the laser radar, the y-axis points to the left of the forward direction, and the z-axis points upward; the camera system is also a three-dimensional coordinate system, the origin of which coincides with the geometric center O of the camera, the z-axis points to the forward direction of the camera, the x-axis points to the right of the forward direction, and the y-axis points downward.

[0055] See also Figure 2 and 3 , point A′3D and point B′ 3D With point A 2D and point B 2D The corresponding relationship is as follows: The homogeneous coordinates A of the upper left corner of the known two-dimensional detection box 2D =(p x1 ,p y1 ,1) T and the upper right corner point B 2D =(p x2 ,p y2 ,1) T , the camera's intrinsic parameter matrix M and the laser radar system To the camera system The coordinate system transformation matrix The constant λ can be set to 100, then we get These two rays can be used to qualitatively analyze the accuracy of the external parameter matrices of the laser radar and camera. If the matrix accuracy is low, the two rays will deviate from the corner point A of the 3D detection frame respectively. 3D and corner point B 3D Among them, the corner point A 2D and point B 2D Point A 3D and point B 3D The image point on the camera image plane.

[0056] In a possible implementation, the data augmentation of the training data includes: performing one or more of the following data augmentation operations on the training data: detection instance augmentation, point cloud scaling, point cloud translation, and point cloud rotation.

[0057] Specifically, by performing data augmentation operations on the training data, the generalization and convergence of the model are improved.

[0058] In a possible implementation, a three-dimensional target detection method is provided, in which a sample to be detected is input into a three-dimensional target detection model optimized by the three-dimensional target detection model optimization method described above, to obtain a three-dimensional target detection result of the sample to be detected.

[0059] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0060] See also Figure 4 In another embodiment of the present invention, a three-dimensional target detection model optimization system is provided, which can be used to implement the above-mentioned three-dimensional target detection model optimization method. Specifically, the three-dimensional target detection model optimization system includes a data acquisition module, an anomaly determination module and a model optimization module.

[0061] Among them, the data acquisition module is used to obtain training data, and train the three-dimensional target detection model N times with the training data to obtain N pre-trained three-dimensional target detection models; the anomaly determination module is used to obtain the positioning confidence variance of each detection frame of each pre-trained three-dimensional target detection model, and obtain the anomaly detection frame according to the preset positioning confidence variance threshold; the model optimization module is used to place the true value frame corresponding to the anomaly detection frame in several target point cloud frames of the training data without changing the geometric position. When the three-dimensional target detection model exhibits consistent anomaly detection behavior under several target point cloud frames, the three-dimensional target detection model is trained after data augmentation of the training data.

[0062] All relevant contents of each step involved in the aforementioned embodiment of the three-dimensional target detection model optimization system can be referred to the functional description of the functional modules corresponding to the three-dimensional target detection model optimization system in the embodiment of the present invention, and will not be repeated here.

[0063] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0064] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the three-dimensional target detection model optimization method.

[0065] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the three-dimensional target detection model optimization method in the above embodiment.

[0066] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0068] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A three-dimensional object detection model optimization method, characterized in that: include: Acquire training data, and train the 3D object detection model N times using the training data to obtain N pre-trained 3D object detection models; Obtaining the positioning confidence variance of each detection frame of each pre-trained 3D object detection model, and obtaining an abnormal detection frame according to a preset positioning confidence variance threshold; The true value box corresponding to the anomaly detection box is placed in several target point cloud frames of the training data without changing its geometric position. When the three-dimensional object detection model exhibits consistent anomaly detection behavior in several target point cloud frames, the three-dimensional object detection model is trained after data augmentation of the training data.

2. The three-dimensional target detection model optimization method according to claim 1, characterized in that: Also includes: By adjusting the length, width, height and geometric center point of the prior box of the three-dimensional target detection model, the distribution of laser points in the prior box or the orientation of the prior box, the changing trend of the intersection ratio of the anomaly detection box and the true value box is obtained, and the length, width, height and geometric center point of the prior box of the three-dimensional target detection model, the distribution of laser points in the prior box or the orientation of the prior box is adjusted according to the changing trend.

3. The three-dimensional target detection model optimization method according to claim 1, characterized in that: Before training the three-dimensional object detection model N times using the training data, the method further includes: The training data is cleaned based on preset cleaning rules; wherein the preset cleaning rules include: obtaining the number of laser points in the true value frame corresponding to each label of each point cloud frame in the training data, and removing the labels corresponding to the true value frames whose number of laser points in the true value frames is less than a preset laser point threshold.

4. The three-dimensional target detection model optimization method according to claim 1, characterized in that: The obtaining of the positioning reliability variance of each detection frame of each pre-trained three-dimensional object detection model and obtaining the abnormal detection frame according to a preset positioning reliability variance threshold comprises: Based on the Euclidean distance between the geometric center points of the detection box and the true value box, a 3D box similarity matrix of each pre-trained 3D object detection model is constructed, and the 3D box similarity matrix of each pre-trained 3D object detection model is processed based on the Hungarian algorithm to obtain a one-to-one pairing relationship between the detection box and the true value box of each pre-trained 3D object detection model, and the detection box paired with the i-th true value box is set as the i-th detection box; and the detection box position confidence of the detection box whose similarity with the paired true value box is lower than a preset similarity threshold is set to 0; The position confidence variance v of the i-th detection box is obtained by the following formula: i : in, is the positioning confidence of the i-th detection box of the n-th pre-trained 3D object detection model; A detection box whose positioning confidence variance is less than a preset positioning confidence variance threshold is an abnormal detection box.

5. The three-dimensional target detection model optimization method according to claim 1, characterized in that: Also includes: The visualization shows the 3D intersection ratio, 2D top view intersection ratio, 2D image intersection ratio, positioning confidence of the detection box, the number of laser points of the detection box, and the number of laser points of the true value box of each pre-trained 3D object detection model.

6. The three-dimensional target detection model optimization method according to claim 1, characterized in that: Also includes: Visualizing the ray OA′ of the 3D object detection model 3D and ray OB′ 3D ; Among them, ray OA′ 3D and ray OB′ 3D Point A′ 3D and point B′ 3D The ray to the camera's optical center O; For LiDAR To the camera system The coordinate system transformation matrix, M is the camera's intrinsic parameter matrix, and λ is a preset constant; A 2D and B 2D A is the top left corner of the 3D detection frame of the 3D target detection model. 3D and the front upper right corner point B 3D The imaging point on the camera image plane; When the OA ′ 3D and ray OB3 ′ D Departure point A 3D and point B 3D When adjusting the external parameter matrix of the laser radar and camera of the 3D target detection model, the ray OA ′ 3D and ray OB3 ′ D Pass point A 3D and point B 3D .

7. The three-dimensional target detection model optimization method according to claim 1, characterized in that: The data augmentation for the training data includes: performing one or more of the following data augmentation operations on the training data: detection instance augmentation, point cloud scaling, point cloud translation, and point cloud rotation.

8. A three-dimensional object detection model optimization system, characterized in that: include: A data acquisition module is used to acquire training data and train the three-dimensional object detection model N times using the training data to obtain N pre-trained three-dimensional object detection models; The anomaly determination module is used to obtain the positioning confidence variance of each detection frame of each pre-trained three-dimensional object detection model, and obtain the anomaly detection frame according to a preset positioning confidence variance threshold; The model optimization module is used to place the true value box corresponding to the anomaly detection box in several target point cloud frames of the training data without changing the geometric position. When the three-dimensional target detection model exhibits consistent anomaly detection behavior under several target point cloud frames, the training data is augmented and then the three-dimensional target detection model is trained.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the three-dimensional object detection model optimization method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the three-dimensional object detection model optimization method as described in any one of claims 1 to 7 are implemented.

Images