Three-view-based point cloud three-dimensional target detection method and medium

Abstract

The invention relates to a point cloud three-dimensional target detection method based on three views and a medium. The method comprises the following steps: acquiring three-dimensional point cloud sample data; a neural network model of a preset framework is trained, the neural network model comprises a preprocessing module, a backbone network model, a coding module and a detection head module, and the preprocessing module carries out voxel division on three-dimensional point cloud sample data to form preset regular three-dimensional voxels; the backbone network model performs feature extraction on regular three-dimensional voxels to obtain a feature map; the detection head module is used for detecting a prediction frame through the feature map; the coding module is used for coding a true value frame in the label to obtain a coding frame, and calculating an intersection-to-union ratio of each prediction frame to the coding frame; the intersection-union ratio is combined with the three-view shape information to obtain a regression quality score of the prediction sample, a loss function is calculated based on the regression quality score, and training is completed when the loss function reaches a preset condition; and detecting based on the trained neural network model.

Description

Technical Field

[0001] The present invention relates to the technical field of computer graphics, and in particular to a point cloud three-dimensional target detection method and medium based on three-view images. Background Art

[0002] Point cloud 3D object detection is one of the main components of 3D object detection tasks and plays a vital role in perceiving the 3D world. Point cloud 3D object detection perceives the 3D world by locating and classifying objects in point cloud scenes, which is crucial to the development of applications such as autonomous driving, intelligent robots, and intelligent transportation.

[0003] In the existing technology, the three-dimensional point cloud data is input into the pre-trained neural network model to locate and classify the target. For example, in recent years, as researchers continue to advance the point cloud target detection task, many excellent works have been produced, such as Pointpillars, which speeds up the three-dimensional target detection in point cloud scenes and enhances the application prospects of this task. VoxelNeXt further increases the network learning ability and improves the accuracy of target detection in point cloud scenes. SIoU learns the shape information of the object to be detected from a single view to improve the accuracy of detection frame regression. The three-view image is composed of three mutually orthogonal projection images, including three basic views: front view, left view and top view. In the field of engineering drawing, the three-view image is often used to depict the three-dimensional shape of geometric objects, which is closely related to the task of regressing the shape of objects in point cloud three-dimensional target detection. Therefore, in the point cloud three-dimensional target detection task, using the three-view idea to obtain the shape information of three-dimensional objects plays an important role in accurately regressing the detection frame size.

[0004] Although, based on previous work on point cloud 3D object detection, it has become possible to detect 3D objects in many real point cloud scenes. However, there are still some bottlenecks to be solved to further improve the accuracy of point cloud 3D object detection. The core issues are: (1) The inability to stably achieve high-precision detection box regression is the key bottleneck that currently restricts the improvement of 3D object detection performance; (2) The lack of 3D object shape information fully learned from the three-view perspective causes deviations in detection box regression. These challenges are particularly prominent in large-scale point cloud scenes, and sparse LiDAR data further exacerbates the difficulty of object positioning.

[0005] Based on the above analysis, a new method is urgently needed to further improve the accuracy and versatility of three-dimensional object detection in point cloud scenes. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a point cloud three-dimensional target detection method based on three views, in which a new intersection-over-union (IoU) calculation method is designed as a loss function to supervise the detection box regression task of the point cloud target detection model and output a point cloud scene with a three-dimensional detection box, so as to improve the adaptive learning ability of the target detection model in the point cloud scene for objects of different shapes, avoid the misrecognition of special-shaped objects, and thus be applied in the field of point cloud three-dimensional target detection technology based on three views to improve the accuracy of target detection.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present application provides a point cloud three-dimensional target detection method based on three views, the method comprising:

[0009] (1) Obtaining labeled 3D point cloud sample data;

[0010] (2) Based on the labeled three-dimensional point cloud sample data, a neural network model with a preset architecture is trained, wherein the neural network model includes a preprocessing module, a backbone network model, an encoding module, and a detection head module, wherein:

[0011] The preprocessing module performs voxel division on the three-dimensional point cloud sample data to form preset regular three-dimensional voxels;

[0012] The backbone network model extracts features from regular three-dimensional voxels to obtain a feature map;

[0013] The detection head module detects the prediction box through the feature map;

[0014] The encoding module encodes the true value frame in the label to obtain the encoding frame, calculates the intersection-and-union ratio of each prediction frame and the encoding frame, calculates the loss function based on the intersection-and-union ratio, and completes the training of the neural network model when the loss function meets the preset condition;

[0015] (3) Acquire the three-dimensional point cloud data to be predicted, and detect the three-dimensional point cloud data to be predicted based on the trained neural network model.

[0016] In one implementation, the preprocessing module inputs the original point cloud of the three-dimensional point cloud sample data and the corresponding true value label into the network, and uses voxel feature encoding to divide the irregular point cloud into a regular three-dimensional grid form.

[0017] In one implementation, the backbone network model uses a feature pyramid network based on three-dimensional sparse convolution to extract multi-scale features of the point cloud, and then compresses the three-dimensional features into a two-dimensional feature map.

[0018] In one implementation, the detection head module outputs the center point coordinates, length, width, height, rotation angle and category information of all predicted boxes for each frame of point cloud.

[0019] In one implementation, the encoding module, if a method without an anchor box type is used, directly uses the truth box; if an anchor box type method is used, encoding is performed according to the following paradigm:

[0020] x t =x g -x a ,y t =y g -y a ,z t =z g -z a ,

[0021] θ t =θ g -θ a ,l t =log l g ,w t = log w g ,h t = log h g ,

[0022] The center coordinates (x, y, z), length (l), width (w), height (h), and rotation angle (θ) of each frame are represented by (x s ,y s ,z s ,l s ,w s ,h s ,θ s ), s∈{g,t,a,p}, g represents the true value box, t represents the target box, a represents the anchor box, and p represents the predicted box.

[0023] In one implementation, for each coding frame, the first K prediction frames with the highest confidence scores are selected from all prediction frames that match it, and the intersection-over-union ratio between each coding frame T and the prediction frame P is calculated as:

[0024]

[0025] Among them, area(T∩P) means calculating the area of ​​the intersection between two boxes, area(T∪P) means calculating the area of ​​the union between two boxes, Int 4D Represents the intersection between two boxes, Vol t and Vol. p Represent the volume of the encoding box and the prediction box respectively, and the intersection is calculated as:

[0026]

[0027] in, It is used to calculate the intersection of the length, width and height of two axis-aligned boxes respectively. The function calculation method is:

[0028]

[0029] It is used to measure the intersection of angle information in the fourth dimension in IoU, and is calculated as follows:

[0030]

[0031] in, Represent the coordinate values ​​of the predicted box angle and the true value box angle on the unit circle respectively;

[0032] The volume calculation method of the prediction box and the encoding box is:

[0033] Vol s = l s × s ×h s ,s∈{p,t}.

[0034] In one implementation, the shape information of the object is collected based on the three perspectives of the three-view image. Left View and top view The calculation method is:

[0035]

[0036] Among them, α c ,c∈{l,w,h} represents the relative value of different dimensions of the encoding frame in all dimensions, and is expressed as Represents the size of the encoding box in three dimensions, α c The calculation method is:

[0037]

[0038] g c ,c∈{l,w,h} represents the prediction error of each dimension, which is calculated as follows:

[0039] g l =|l p -l t | / max(l p ,l t )

[0040] g w =|w p -wt | / max(w p ,w t )

[0041] g h =|h p -h t | / max(h p ,h t ).

[0042] After obtaining the shape information based on the three views, the regression quality score of the prediction box is calculated as follows:

[0043]

[0044] In one implementation, the loss function Including classification loss Regression Loss and anchor box orientation loss The calculation method is: Among them, λ1, λ2, λ3 represent the weights of the loss function.

[0045] In one implementation, λ1 is 1.0, λ2 is 2.0, and λ3 is 0.2.

[0046] In a second aspect, the present application provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the three-view-based point cloud three-dimensional target detection method of the first aspect.

[0047] The present invention adopts the above technical solution, which has the following advantages:

[0048] 1. This paper proposes a novel intersection-over-union (IoU) calculation method to enhance the optimization process of the 3D object detection network by recalibrating the relationship between IoU and angle, and solves the problem of incorrect coupling between IoU and angle.

[0049] 2. The present invention proposes an IoU calculation method, which fuses shape information based on three orthogonal projection views, adaptively calculates the IoU of objects of different categories, and makes full use of the rich shape information inherent in three-dimensional objects.

[0050] 3. The present invention proposes a universal solution. By redefining the anchor point encoding scheme and restating the angle, the proposed method promotes a seamless transition between the anchor box and anchor-free methods, thereby achieving a higher level of universality.

[0051] 4. Applying the present invention to a point cloud scene three-dimensional target detection model based on deep learning can greatly improve the detection performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a schematic diagram of a network structure of a three-dimensional target detection method according to an embodiment of the present invention;

[0053] Figure 2 Schematic diagram of the detection results of the embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0055] In view of the problem that the accuracy of target detection based on three-dimensional point cloud in the prior art needs to be improved, the present application provides a three-dimensional target detection method based on point cloud of three views, the method comprising:

[0056] (1) Obtaining labeled 3D point cloud sample data;

[0057] (2) Based on the labeled three-dimensional point cloud sample data, a neural network model with a preset architecture is trained, wherein the neural network model includes a preprocessing module, a backbone network model, an encoding module, and a detection head module, wherein:

[0058] The preprocessing module performs voxel division on the three-dimensional point cloud sample data to form preset regular three-dimensional voxels;

[0059] The backbone network model extracts features from regular three-dimensional voxels to obtain a feature map;

[0060] The detection head module detects the prediction box through the feature map;

[0061] The encoding module encodes the true value frame in the label to obtain the encoding frame, calculates the intersection-and-union ratio of each prediction frame and the encoding frame, calculates the loss function based on the intersection-and-union ratio, and completes the training of the neural network model when the loss function meets the preset condition;

[0062] (3) Acquire the three-dimensional point cloud data to be predicted, and detect the three-dimensional point cloud data to be predicted based on the trained neural network model.

[0063] In conjunction with the drawings of the present application, a method for calculating the intersection over union (IoU) is involved in point cloud three-dimensional target detection based on three views.

[0064] A shape-adaptive IoU calculation method based on three-view shape information guidance in point cloud target detection includes the following steps:

[0065] 1) Obtain the input point cloud and labels of the network, divide the point cloud into voxels, and form a regular 3D voxel format;

[0066] 2) Inputting the regular point cloud processed in 1) into the backbone network, and obtaining the feature map of the input point cloud through the feature extraction network;

[0067] 3) Input the point cloud feature map extracted in 2) into the detection head module to obtain the information of each dimension of the prediction box;

[0068] 4) If the anchor box type method is used, the true value box in the label needs to be specifically encoded. If the anchor box type method is used, no encoding is required and the true value box is used directly;

[0069] 5) According to the prediction information in 3), calculate the intersection and union ratio of each prediction box and the encoding box (or true value box);

[0070] 6) According to the intersection-union ratio obtained in 5), the quality score of the predicted sample is calculated in combination with the shape information of the three views;

[0071] 7) Calculate the overall loss of the sample according to the sample quality score in 6);

[0072] 8) After the overall loss of the neural network model tends to be stable, the neural network weights for 3D object detection are obtained;

[0073] 9) Input the point cloud scene to be detected, predict the target in the point cloud scene through the saved training weights, and output the positioning box and target category of the target.

[0074] Furthermore, in step 1), the original point cloud and the corresponding true value labels are input into the network, and the irregular point cloud is divided into a regular three-dimensional grid using voxel feature encoding.

[0075] Furthermore, in step 2), a feature pyramid network based on three-dimensional sparse convolution is used to extract multi-scale features of the point cloud, and then the sparse three-dimensional features are compressed into a dense two-dimensional feature map.

[0076] Furthermore, in step 3), for each frame of point cloud, the center point coordinates, length, width, height, rotation angle and category information of all predicted boxes are output.

[0077] Furthermore, in step 4), if the method without anchor box is used, the true value box is used directly. If the method with anchor box is used, encoding is performed according to the following high-flexibility encoding paradigm:

[0078] y t =xg -x a ,y t =y g -y a ,z t =z g -z a ,

[0079] θ t =θ g -θ a ,l t =log l g ,w t = log w g ,h t = log h g ,

[0080] The center coordinates (x, y, z), length (l), width (w), height (h), and rotation angle (θ) of each frame are represented by (x s ,y s ,z s ,l s ,w s ,h s ,θ s ), s∈{g,t,a,p}, g represents the true value box, t represents the target box, a represents the anchor box, and p represents the predicted box.

[0081] Furthermore, in step 5), for each coding frame (or true value frame), the first K prediction frames with the highest confidence scores are selected from all prediction frames matching it, and K is 500. The intersection-over-union ratio between each coding frame T and the prediction frame P is calculated as:

[0082]

[0083] Among them, area(T∩P) means calculating the area of ​​the intersection between two boxes, and area(T∪P) means calculating the area of ​​the union between two boxes. 4D Represents the intersection between two boxes, Vol t and Vol. p Represent the volume of the encoding box and the prediction box respectively, and the intersection is calculated as:

[0084]

[0085] in, It is used to calculate the intersection of the length, width and height of two axis-aligned boxes respectively. The function calculation method is:

[0086]

[0087] It is used to measure the intersection of angle information in the fourth dimension in IoU, and is calculated as follows:

[0088]

[0089] in, Represent the coordinate values ​​of the predicted box angle and the true value box angle on the unit circle respectively.

[0090] The volume calculation method of the prediction box and the encoding box is:

[0091] Vol s = l s × s ×h s ,s∈{p,t}.

[0092] Furthermore, in step 6), the shape information of the object is collected based on the three perspectives of the three-view drawing, the front view Left View and top view The calculation method is:

[0093]

[0094] Among them, α c ,c∈{l,w,h} represents the relative value of different dimensions of the encoding frame in all dimensions, and is expressed as Represents the size of the encoding box in three dimensions, α c The calculation method is:

[0095]

[0096] g c ,c∈{l,w,h} represents the prediction error of each dimension, which is calculated as follows:

[0097] g l =|l p -l t | / max(l p ,l t )

[0098] g w =|w p -w t | / max(w p ,w t )

[0099] g h =|h p -h t | / max(h p ,h t ).

[0100] After obtaining the shape information based on the three views, the regression quality score of the prediction box is calculated as follows:

[0101]

[0102] Further, in step 7), taking the three-dimensional object detection method with anchor box as an example, the overall loss Including classification loss Regression Loss and anchor box orientation loss The calculation method is: Among them, λ1, λ2, and λ3 represent the weights of the loss function, λ1 is 1.0, λ2 is 2.0, and λ3 is 0.2.

[0103] The classification loss Quality Focal Loss loss, anchor box direction loss Calculated by CrossEntropy Loss, regression loss The TV4DIoU method proposed in the present invention can be used, and the calculation method is:

[0104]

[0105] The present method is further illustrated in the following with reference to a specific application example.

[0106] A general special IoU calculation method and system, specifically a deep learning method applied in three-dimensional object detection in point cloud scenes. It includes: an angle measurement method to repair the coupling relationship between IoU and angle errors, TV4DIoU guided by three-view information, and a solution to improve the versatility of TV4DIoU. The implementation steps are: input the pre-processed point cloud and labels into the network for pre-training; perform data enhancement processing such as random flipping, rotation and downsampling on all data, and input into the deep neural network to calculate the prediction loss; add up all the calculated losses as the final loss; input the point cloud scene to be detected, use the weights obtained from the training network, and mark the positioning box and category of the target point cloud.

[0107] In one embodiment of the present invention, based on a three-dimensional target detection method VoxelNeXt without an anchor box, the highly versatile TV4DIoU series method proposed in the present invention is combined. This embodiment uses the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0108] 1) In this embodiment, we use the nuScenes point cloud dataset (nuscenes: A multimodal dataset for autonomous driving). This dataset collects 1,000 point cloud scenes with labeled samples, including 10 types of targets. Each scene is about 20 seconds, and key frames are selected at a frequency of 2Hz for annotation. Among them, the training set contains 700 scenes, and the validation set and test set each contain 150 scenes;

[0109] 2) If Figure 1 As shown, the training set is used as input, and the point cloud is first subjected to data enhancement such as random flipping, rotation, and scaling;

[0110] 3) Divide the irregular point cloud in 2) into regular three-dimensional voxels, and the size of the three-dimensional voxels is (0.1, 0.1, 0.15);

[0111] 4) encoding the point cloud in the voxel in 3) into voxel features through an average voxel feature encoding module;

[0112] 5) The voxel features in 4) are transmitted to the three-dimensional backbone network for training. The type of backbone network can be selected according to the needs. In this example, a multi-layer convolutional neural network based on three-dimensional sparse convolution is used, which contains 6 layers of three-dimensional sparse convolution;

[0113] 6) Reduce the 3D features in 5) into a 2D feature map through 2D sparse convolution;

[0114] 7) Input the feature map in 6) into the detection head, and output the center point coordinates, length, width, height, rotation angle and category confidence of the detection box respectively;

[0115] 8) The category confidence and true value label in 7) are used to calculate the classification loss through the classification loss branch;

[0116] 9) Input the regression information of the detection box in 7) and the true value box into the TV4DIoU proposed by the present invention, calculate the intersection-over-union ratio, and obtain the regression loss;

[0117] 10) Add the loss functions obtained in 8) and 9), set λ1 and λ2 to 1.0 and 2.0 respectively, and get the overall loss;

[0118] 11) Input the overall loss in 10) into the optimizer, use the Adam Onecycle optimizer to adjust the model parameters and update the learning rate, back-propagate the improved parameters to the network, and obtain the network weights after training for 12 epochs.

[0119] 12) Fix the network weights in 11), input the validation set or the test set, and output the three-dimensional object detection results.

[0120] In this embodiment, the hardware environment configuration includes an AMD R9 5950X processor with 16 cores and a Geforce RTX 3090 graphics card with 24GB of video memory. In terms of software, it runs on the Ubuntu 24.04 operating system, with CUDA version 11.3, using Pytorch 1.10 as the deep learning framework, and using OpenPCDet as the toolset. In the training configuration, the starting value of the learning rate is set to 0.003, and it goes through 12 iteration cycles. It is worth noting that according to different target detection algorithms and data set characteristics, the parameters in other embodiments can be appropriately tuned. After training, the obtained network weights can be applied to the testing phase, and these preset weights are used to classify and locate the input point cloud data.

[0121] In summary, the present invention can realize the detection of three-dimensional targets in autonomous driving scenes.

[0122] To verify the effectiveness and practicality of the method proposed in the present invention, an example on the nuScenes dataset is given below. Table 1 shows the detection results of the example on the validation set. The various measurement standards are AP (Average Precision), mAP (mean Average Precision) is the average AP value of all categories, and NDS (Nuscenes Detection Score) is a comprehensive indicator proposed by the nuScenes dataset.

[0123]

[0124] Table 1

[0125] As shown in Table 1, after adding the three-view intersection-and-union ratio calculation method (TV4DIoU) proposed in the present invention, the detection performance of the VoxelNeXt model is greatly improved, proving the effectiveness of the present invention. Figure 2 The visual detection results shown in the figure show that after adding the method proposed by the present invention, the object to be detected is accurately detected and has a precise position and shape. The method proposed by the present invention can also be flexibly applied to other three-dimensional object detection network frameworks.

[0126] In an embodiment of the present application, a computer-readable storage medium is also provided accordingly, in which a computer program is stored. When a computer device executes the computer program, the method in the embodiment of the present application is implemented.

[0127] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0128] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0129] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (Processor) to perform some steps of the above-mentioned method of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0130] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A point cloud 3D object detection method based on three views, It is characterized in that The method comprises: (1) Obtaining labeled 3D point cloud sample data; (2) Based on the labeled three-dimensional point cloud sample data, a neural network model with a preset architecture is trained, wherein the neural network model includes a preprocessing module, a backbone network model, an encoding module, and a detection head module, wherein: The preprocessing module performs voxel division on the three-dimensional point cloud sample data to form preset regular three-dimensional voxels; The backbone network model extracts features from regular three-dimensional voxels to obtain a feature map; The detection head module detects the prediction box through the feature map; The encoding module encodes the true value frame in the label to obtain the encoding frame, calculates the intersection-and-union ratio of each prediction frame and the encoding frame, calculates the loss function based on the intersection-and-union ratio, and completes the training of the neural network model when the loss function meets the preset condition; (3) Acquire the three-dimensional point cloud data to be predicted, and detect the three-dimensional point cloud data to be predicted based on the trained neural network model.

2. The point cloud three-dimensional target detection method based on three views according to claim 1, It is characterized in that The preprocessing module inputs the original point cloud of the three-dimensional point cloud sample data and the corresponding true value label into the network, and uses voxel feature encoding to divide the irregular point cloud into a regular three-dimensional grid form.

3. The point cloud three-dimensional target detection method based on three views according to claim 1, It is characterized in that The backbone network model uses a feature pyramid network based on three-dimensional sparse convolution to extract multi-scale features of point clouds, and then compresses the three-dimensional features into a two-dimensional feature map.

4. The method for detecting three-dimensional objects based on point clouds of three views according to claim 1, It is characterized in that The detection head module outputs the center point coordinates, length, width, height, rotation angle and category information of all predicted boxes for each frame of point cloud.

5. The point cloud 3D target detection method based on three views according to claim 4, It is characterized in that The encoding module, if the method without anchor box type is used, directly uses the truth box; if the method with anchor box type is used, encoding is performed according to the following paradigm: x t =x g -x a ,y t =y g -y a ,z t =z g -z a , i t =θ g -θ a ,l t =log l g ,w t =log w g ,h t =log h g , The center coordinates (x, y, z), length (l), width (w), height (h), and rotation angle (θ) of each frame are represented by (x s ,y s ,z s ,l s ,w s ,h s ,θ s ), s∈{g,t,a,p}, g represents the true value box, t represents the target box, a represents the anchor box, and p represents the predicted box.

6. The method for detecting three-dimensional objects in point cloud based on three views according to claim 5, It is characterized in that For each coding frame, select the first K prediction frames with the highest confidence scores from all the prediction frames that match it. The intersection-over-intersection ratio between each coding frame T and the prediction frame P is calculated as: Among them, area(T∩P) means calculating the area of ​​the intersection between two boxes, area(T∪P) means calculating the area of ​​the union between two boxes, Int 4D Represents the intersection between two boxes, Vol t and Vol. p Represent the volume of the encoding box and the prediction box respectively, and the intersection is calculated as: in, It is used to calculate the intersection of the length, width and height of two axis-aligned boxes respectively. The function calculation method is: It is used to measure the intersection of angle information in the fourth dimension in IoU, and is calculated as follows: in, Represent the coordinate values ​​of the predicted box angle and the true value box angle on the unit circle respectively; The volume calculation method of the prediction box and the encoding box is: Vol s =l s ×w s ×h s ,s∈{p,t}。 7. The method for detecting three-dimensional objects in point cloud based on three views according to claim 6, It is characterized in that The shape information of the object is collected from three perspectives of the three-view drawing. Left View and top view The calculation method is: Among them, α c ,c∈{l,w,h} represents the relative value of different dimensions of the encoding frame in all dimensions, and is expressed as Represents the size of the encoding box in three dimensions, α c The calculation method is: g c ,c∈{l,w,h} represents the prediction error of each dimension, which is calculated as follows: g l =|l p -l t | / max(l p ,l t ) g w =|w p -w t | / max(w p ,w t ) g h =|h p -h t | / max(h p ,h t ) After obtaining the shape information based on the three views, the regression quality score of the prediction box is calculated as follows:

8. The method for detecting three-dimensional objects in point cloud based on three views according to claim 7, It is characterized in that Loss Function Including classification loss Regression Loss and anchor box orientation loss The calculation method is: Among them, λ 1 ,λ 2 ,λ 3 Represents the weight of the loss function.

9. The method for detecting three-dimensional objects in point cloud based on three views according to claim 8, It is characterized in that λ 1 Take 1.0, λ 2 Take 2.0, λ 3 Take 0.

2.

10. A computer-readable storage medium, It is characterized in that A computer program is stored, and the computer program is executed by a processor to implement the point cloud three-dimensional target detection method based on three views as described in any one of claims 1 to 9.

Images