Three-dimensional target detection method and system based on a votenet model

Abstract

A 3D target detection method based on a votenet model includes: constructing a votenet model; constructing a point cloud dataset for training the votenet model according to the target of interest; constructing a seed point displacement loss function based on a two-layer nested 3D rectangular box space partition for training the votenet model; constructing other loss functions for training the votenet model based on the original method of the votenet model, including a foreground-background classification loss function, a center offset loss function, a dimension offset loss function, and an orientation angle offset loss function; training the votenet model based on the constructed point cloud dataset and loss function; acquiring point cloud data of a scene to be detected using an RGB-D camera; and outputting a 3D target detection result of the target of interest through the votenet model based on the point cloud data of the scene to be detected. The present invention can effectively reduce the false positive rate of 3D target detection results without increasing model inference delay.

Description

[Technical Field]

[0001] This invention relates to the field of indoor three-dimensional target detection, and more particularly to a three-dimensional target detection method based on the votet model. [Background technology]

[0002] Target detection algorithms have a wide range of applications and continue to attract attention in the field of computer vision research. In recent years, with the development of deep learning, research on target detection in images has made tremendous progress. Compared to two-dimensional target detection technology, three-dimensional target detection technology can be linked with depth information and provide spatial scene information such as the position, orientation, and size of the target, and is rapidly developing in the fields of autonomous driving and robotics. Currently, the detection recall rate of three-dimensional target detection methods based on the votetet model is relatively high, but seed points on non-target surfaces generated by the votetet model tend to appear in the region between multiple targets after displacement processing. If the distance between targets is too close, features of different targets get mixed into neighboring points where several aggregate points generated by the votetet model converge. Although the features as foregrounds of the aggregated features are clear, the positioning accuracy is extremely low, resulting in the generation of highly reliable false detection targets. Existing methods usually reduce the false detection rate of the detection results by performing refinement processing on the detection results, but it is unavoidable that extra refinement processing leads to extra computation time, thus increasing inference time. Suppressing the occurrence of false positives without unnecessarily increasing inference time is a major challenge facing 3D target detection technology. [Overview of the project]

[0003] The objective of the present invention is to provide a three-dimensional target detection method based on the votetet model to address the shortcomings of the prior art. Compared to other three-dimensional target detection methods based on the votetet model, this method does not increase model inference time, has a low false positive rate, and a high detection recall rate.

[0004] The object of the present invention is achieved by the following technical method: a three-dimensional target detection method based on a votenet model, comprising the following steps.

[0005] Step 1: Build the votet model.

[0006] Step 2: Construct a point cloud dataset to train the votet model, tailored to your target of interest.

[0007] Step 3: Construct a seed point displacement loss function based on a two-layer nested three-dimensional rectangular box spatial partitioning to train the votet model.

[0008] Step 4: Based on the original method of the votet model, construct additional loss functions for training the votet model, including foreground-background classification loss function, center offset loss function, dimension offset loss function, and direction angle offset loss function.

[0009] Step 5: Train the votet model based on the constructed point cloud dataset and loss function.

[0010] Step 6: Use an RGB-D camera to acquire point cloud data of the scene to be detected.

[0011] Step 7: Based on the point cloud data of the scene to be detected, the 3D target detection results of the target of interest are output through the votet model.

[0012] Furthermore, in Step 1, we construct the votetet model based on the construction method described in the votetet paper.

[0013] Furthermore, in step 2, the above point cloud dataset refers to a plurality of point cloud data samples including the target of interest and annotation information of the three-dimensional rectangular bounding box of each target of interest in the sample. The point cloud data may be from various types of sensors.

[0014] Furthermore, in step 3 above, for each training sample, the seed point displacement loss function is realized by the following sub-steps.

[0015] (3.1) Input the point cloud data of the training sample into the votenet model to obtain the seed point coordinates extracted by the votenet model and the coordinate offsets of each seed point predicted by the votenet model.

[0016] (3.2) Assume that the training sample contains a total of n t targets, and for the i-th target where i ∈ [1, n t , let the actual three-dimensional rectangular bounding box of the target be Box i =(x i , y i , z i , l i , w i , h i, θ i ), where x i , y i , z i are the center point coordinates of the bounding box, l i , w i , h i are the length, width, and height of the bounding box, θ i is the orientation angle of the bounding box. For each actual three-dimensional rectangular bounding box Box i , the center point coordinates and orientation angle of the bounding box are the same, but for the three-dimensional rectangular box Box i (e) =(x i , y i , z i , k*l i , k*wi ,k*h i ,θ i ) generates.

[0017] (3.3) The total number of seed points is n s Then, traversing the entire seed point, j∈[1,n s ]th seed point P j Regarding seed point P j The coordinates in three-dimensional space are (x j ,y j ,z j ) and seed point P j A real three-dimensional rectangular bounding box for the entire target. i i∈[1,n t Traverse ] and seed point P j coordinates (x j ,y j ,z j ) A real three-dimensional rectangular bounding box Box i If the seed point is on the interior or exterior surface, there is one corresponding target point T for that seed point. j Generates the target point T. j The coordinate values ​​are those of a real three-dimensional rectangular bounding box. i Center point coordinates (x i ,y i ,z i ) is equal to seed point P j coordinates (x j ,y j ,z j If the target point T lies inside or outside the surface of multiple real three-dimensional rectangular bounding boxes, j The coordinate values ​​are those of the seed point P, which is the center point of the actual three-dimensional rectangular bounding box described above. j It is equal to the coordinates of the center point of the nearest three-dimensional rectangular bounding box.

[0018] (3.4) The number of seed points that do not have a corresponding target point is n' s Then, traverse all seed points that do not have a corresponding target point, and j∈[1,n' sThe j th seed point P j , the three-dimensional rectangular bounding box Box i (e) after overall expansion according to it, i ∈ [1, n t is traversed, and if the coordinates (x j , y j , z j ) of the seed point P j are inside or on the outer surface of the expanded three-dimensional rectangular bounding box Box i (e) , one corresponding target point V j is generated for the seed point. The coordinate values of the target point V j are equal to the coordinate values (x j , y j , z j ) of the seed point P j .

[0019] (3.5) Traverse all seed points, and for the s th seed point P j , if the coordinate offset predicted by the votenet model for the seed point is (Δx j , Δy j , Δz j ), and there is a corresponding target point T j or target point V j j for the seed point P j , one corresponding prediction point Q j is generated for the seed point. The coordinate values of the prediction point Q j are (x j + Δx j , y j + Δy j , z j + Δz j ).)

[0020] (3.6) Calculate the displacement loss L vote-reg of the seed point. PT k is the target point T k existing seed point Pk and the total number of PT k is n T Let PV m be the target point V m where the seed point P exists m Let the total number of PV m be n V If D(A,B) is the distance between point A and point B in three-dimensional space and α is the balance factor, the specific calculation formula for the displacement loss of the seed point is as follows JPEG0007897658000001.jpg41122

[0021] The present invention further includes a three-dimensional target detection system based on the votenet model including the following

[0022] a votenet model construction module for constructing a votenet model

[0023] a point cloud dataset construction module for constructing a point cloud dataset for training the votenet model according to the target of interest

[0024] a seed point displacement loss function construction module for constructing a seed point displacement loss function based on a two-layer nested three-dimensional rectangular box space division for training the votenet model

[0025] another loss function construction module for constructing other loss functions for training the votenet model, including a foreground-background classification loss function, a center offset loss function, a dimension offset loss function, and a direction angle offset loss function, based on the original method of the votenet model

[0026] a votenet model training module for training the votenet model based on the constructed point cloud dataset and loss function

[0027] A point cloud data acquisition module for acquiring point cloud data of a scene to be detected using an RGB-D camera.

[0028] A result output module for outputting 3D target detection results of a target of interest through a votet model, based on point cloud data of the scene to be detected.

[0029] The present invention further includes a computer-readable storage medium that stores a program which, when executed by a processor, realizes a three-dimensional target detection method based on the votet model of the present invention.

[0030] The present invention provides a method for determining the three-dimensional spatial positioning of indoor targets, specifically obtaining the position coordinates of a particular target within the camera's field of view in three-dimensional space. By constructing a seed point displacement loss calculation method based on a two-layer nested three-dimensional rectangular box spatial partitioning, the invention prevents seed points located in the background region near the target in the votent model from moving to the vicinity of the target's central region during the displacement phase. This avoids the votent model generating candidate boxes that have high confidence in the vote point set stage but low match to the target. Compared to other three-dimensional target detection methods based on votent models, the present invention can effectively reduce the false positive rate of three-dimensional target detection results without increasing model inference delay.

[0031] The beneficial effect of this invention lies in reducing the probability that background seed points in the votetet model are displaced between multiple targets, thereby reducing the false positive rate of the detection results. Compared to seed point partitioning methods based on single-layer three-dimensional rectangular boxes, this method effectively avoids error monitoring signals due to incomplete annotations of samples, further improving the recall rate of the detection results. This invention solves the problem of the high false positive rate of current three-dimensional target detection methods based on votetet models, while not introducing extra computational load into the votetet model inference stage and not increasing the model inference time. [Brief explanation of the drawing]

[0032] [Figure 1] Figure 1 is a flowchart of the three-dimensional target detection method of the present embodiment, as shown in the exemplary example. [Figure 2] Figure 2 is a schematic diagram of the target three-dimensional rectangular bounding box shown in the exemplary embodiment. [Figure 3] Figure 3 is a schematic diagram showing the depth map shown in the exemplary embodiment converted into point cloud data. [Figure 4] Figure 4 shows a method for realizing the seed point displacement loss function based on a two-layer nested three-dimensional rectangular box spatial partitioning, as demonstrated in an exemplary embodiment. [Figure 5] Figure 5 is a schematic diagram of the system structure of the present application as shown in an exemplary embodiment. [Modes for carrying out the invention]

[0033] The present invention will be described in more detail below, in conjunction with specific examples. It should be understood that the specific examples described herein are merely illustrative and not intended to limit the invention.

[0034] Figure 1 is a schematic diagram of a three-dimensional target detection method for chair targets in one embodiment of the present invention, and includes the following steps.

[0035] Step S101: Build the votet model.

[0036] The above votenet model is a votenet model found in a publicly published votenet paper, and preferably a votenet model constructed directly based on the construction method described in the votenet paper.

[0037] Step S102: Construct a point cloud dataset for training the votet model, tailored to the chair targets.

[0038] The above-mentioned point cloud dataset refers to multiple point cloud data samples containing chair-like targets and annotation information for the three-dimensional rectangular bounding boxes of each chair-like target in the samples. Preferably, the publicly available SUN RGB-D dataset can be used directly.

[0039] In an exemplary embodiment, as shown in Figure 2, the three-dimensional rectangular bounding box must completely enclose the target in three-dimensional space.

[0040] Step S103: Construct a seed point displacement loss function based on a two-layer nested three-dimensional rectangular box spatial partitioning for training the votet model.

[0041] Step S104: Construct additional loss functions for training the votet model.

[0042] The other loss functions mentioned above used to train the votenet model refer to the foreground-background classification loss function, center offset loss function, dimension offset loss function, and direction angle offset loss function of the votenet model, and preferably these are foreground-background classification loss functions, center offset loss function, dimension offset loss function, and direction angle offset loss function that construct the votenet model directly based on the construction method described in the votenet paper.

[0043] Step S105: Train the votet model based on the constructed point cloud dataset and loss function.

[0044] The above training refers to obtaining optimized votenet model parameters using a backpropagation algorithm, and preferably involves implementing the votenet model parameter update process using an SGD algorithm.

[0045] Step S106: Use an RGB-D camera to acquire point cloud data of the scene to be detected.

[0046] The point cloud data mentioned above refers to the point cloud data generated by projecting the pixels of the depth image produced by the RGB-D camera into three-dimensional space, based on the intrinsic parameters of the RGB-D camera. In one exemplary embodiment, as shown in Figure 3, the depth image produced by the RGB-D camera is projected into three-dimensional space to form point cloud data.

[0047] Step S107: Based on the point cloud data of the scene to be detected, the 3D target detection results for chairs are output through the votet model.

[0048] In one exemplary embodiment, as shown in Figure 4, the steps for calculating the seed point displacement loss function based on a two-layer nested three-dimensional rectangular box spatial partitioning for each training sample are as follows:

[0049] Step S201: Input the point cloud data of the training sample into the votet model to obtain the seed point coordinates extracted by the votet model and the coordinate offset of each seed point predicted by the votet model.

[0050] Step S202: The training sample consists of a total of n t Let i ∈ [1, n] contain a target. t For the [th]th target, the actual three-dimensional rectangular bounding box of the target is Box i =(x i ,y i ,z i ,l i ,wi ,h i, θ i ) and among them, x i ,y i ,z i l is the coordinate of the center point of the bounding box. i ,w i ,h i θ is the length, width, and height of the bounding box. i This is the direction angle of the bounding box, and represents the actual three-dimensional rectangular bounding box of the target. i Each of these bounding boxes has the same center point coordinates and direction angle, but its length, width, and height are each scaled up by k times. i (e) =(x i ,y i ,z i ,k*l i ,k*w i ,k*h i ,θ i ) generates.

[0051] The above is a three-dimensional rectangular box enlarged by k times. i (e) In this case, k is generally a real number between [1,2], and preferably k=1.5.

[0052] Step S203: The total number of seed points is n s Then, traversing the entire seed point, j∈[1,n s ]th seed point P j Regarding seed point P j The coordinates in three-dimensional space are (x j ,y j ,z j ) and seed point P j A real three-dimensional rectangular bounding box for the entire target. i i∈[1,n t Traverse ] and seed point P j coordinates (x j ,y j ,z j) A real three-dimensional rectangular bounding box Box i If the seed point is on the interior or exterior surface, there is one corresponding target point T for that seed point. j Generates the target point T. j The coordinate values ​​are those of a real three-dimensional rectangular bounding box. i Center point coordinates (x i ,y i ,z i ) is equal to seed point P j coordinates (x j ,y j ,z j If the target point T lies inside or outside the surface of multiple real three-dimensional rectangular bounding boxes, j The coordinate values ​​are those of the seed point P, which is the center point of the actual three-dimensional rectangular bounding box described above. j It is equal to the coordinates of the center point of the nearest three-dimensional rectangular bounding box.

[0053] Step S204: The number of seed points that do not have a corresponding target point is n'. s Then, traverse all seed points that do not have a corresponding target point, and j∈[1,n' s ]th seed point P j In contrast, seed point P j The three-dimensional rectangular bounding box after overall expansion. i (e) i∈[1,n t Traverse ] and seed point P j coordinates (x j ,y j ,z j ) A three-dimensional rectangular bounding box after expansion i (e) If the seed point is on the interior or exterior surface, there is one corresponding target point V j Generates the target point V. j The coordinate values ​​are those of seed point P. j The coordinate values ​​(x j ,y j ,z j It is equal to ).

[0054] Step S205: Traverse the entire seed point, and j∈[1,n s ]th seed point P j For this, the coordinate offset that the votet model predicts for the seed point is (Δx j ,Δy j ,Δz j ) and seed point P j The corresponding target point T j Or target point V j If a seed point P exists, j For this purpose, one corresponding prediction point Q j Generates the predicted point Q. j The coordinate values ​​are (x j +Δx j ,y j +Δy j ,z j +Δz j )

[0055] Step S206: Seed point displacement loss L vote-reg Calculate PT k target point T k Seed point P exists k PT k The total number of n T , PV m Target point V m Seed point P exists m , PV m The total number of n V Let D(A,B) be the distance between points A and B in three-dimensional space, and let α be the equilibrium factor. Then, the displacement loss L of the seed point. vote-reg The specific calculation formula is as follows: JPEG0007897658000002.jpg41122

[0056] The above equilibrium factor α is used to equilibrium the displacement losses of different types of seed points, and preferably α = 0.1.

[0057] The present invention further provides a computer-readable storage medium in which a computer program used to perform a three-dimensional target detection method based on the votet model provided in Figure 1 above is stored.

[0058] As shown in Figure 5, the present invention further provides a schematic diagram of a three-dimensional target detection system based on the votetet model corresponding to Figure 1. The three-dimensional target detection system based on the votetet model of the present invention includes the following:

[0059] A Votenet model building module for constructing a Votenet model.

[0060] A point cloud dataset construction module for building point cloud datasets to train votet models tailored to target interests.

[0061] A seed point displacement loss function builder module for training the votenet model, which constructs a seed point displacement loss function based on a two-layer nested three-dimensional rectangular box spatial partitioning.

[0062] Based on the original method of the votenet model, this module constructs additional loss functions for training the votenet model, including foreground / background classification loss functions, center offset loss functions, dimension offset loss functions, and direction angle offset loss functions.

[0063] A votet model building module for training a votet model based on a constructed point cloud dataset and loss function.

[0064] A point cloud data acquisition module for acquiring point cloud data of a scene to be detected using an RGB-D camera.

[0065] A result output module for outputting 3D target detection results of a target of interest through a votet model, based on point cloud data of the scene to be detected.

[0066] As shown in Figure 5, at the hardware level, the three-dimensional target detection system based on the votet model includes a processor, an internal bus, a network interface, internal memory, and non-volatile memory, and of course, may also include other hardware necessary for the operation. The processor realizes the data acquisition method described in Figure 1 above by reading the corresponding computer program from the non-volatile memory into the internal memory and executing it. Of course, the present invention does not exclude other methods besides the software implementation method, such as logical devices or methods of combining software and hardware, meaning that the execution entity of the following processing flow is not limited to each logical unit, but may also be hardware or a logical device.

[0067] An improvement in a technology can be clearly distinguished as either a hardware improvement (e.g., an improvement in circuit structure such as diodes, transistors, or switches) or a software improvement (an improvement in method flow). However, with the advancement of technology, many method flow improvements today can be considered direct improvements to hardware circuit structure. Almost all designers obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be realized using a hardware physical module. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is an integrated circuit whose logic function is determined by the user's programming of the device. Because the designer can "integrate" a single digital system onto a single PLD through their own programming, there is no need to request the design and manufacture of a dedicated integrated circuit chip from a chip manufacturer.Furthermore, today, instead of manually manufacturing integrated circuit chips, this type of program is often implemented by modifying it into "logic compiler" software, which is similar to the software compiler used when developing and writing programs. The original code before compilation must also be written in a specific programming language, which is called a Hardware Description Language (HDL). There is not just one type of HDL, but a wide variety, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). However, the most widely used today are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should know that by performing a little logical programming using one of the aforementioned hardware description languages ​​and programming it within the integrated circuit, it is possible to easily obtain a hardware circuit that realizes the logical method flow.

[0068] A controller can be implemented in any suitable way, such as a microprocessor or processor, a computer-readable medium for storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. A memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will know that a controller can be implemented not only in a purely computer-readable program code manner, but also in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming method steps. Thus, this type of controller can be considered a type of hardware component, and the devices for realizing the various functions contained within it can also be considered structures within the hardware component. Alternatively, it is even possible to consider that the devices for realizing various functions may be software modules that implement the methods, or structures within hardware components.

[0069] The systems, devices, modules, or units described in the above embodiments can be implemented specifically by computer chips or entities, or by products having some function. A typical implementing device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a mobile phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0070] For the sake of convenience in description, the above-mentioned device is described by dividing it into various units according to its function. Of course, when implementing the present invention, the functions of each unit can be realized within the same or multiple software and / or hardware.

[0071] Those skilled in the art will know that embodiments of the present invention can be provided as methods, systems, or computer program products. Thus, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining both software and hardware. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-compatible storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) containing computer-compatible program code.

[0072] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products based on embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. By providing these computer program instructions to the processor of a general-purpose computer, a dedicated computer, an embedded processor, or other programmable data processing device, a device can be generated in which instructions executed by the processor of the computer or other programmable data processing device can realize the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0073] These computer program instructions can also be stored in computer-readable memory that can guide a computer or other programmable data processing device to operate in a particular manner, thereby causing the instructions stored in the computer-readable memory to generate a product that includes an instruction unit that implements functions specified in one or more flows of a flowchart and / or one or more blocks of a block diagram.

[0074] These computer program instructions can also be implemented on a computer or other programmable data processing device, causing a series of operational steps to be executed on the computer or other programmable device to generate the processing that the computer implements, thereby providing steps for the instructions executed on the computer or other programmable device to implement a function specified in one or more flows of a flowchart and / or one or more blocks of a block diagram.

[0075] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and internal memory.

[0076] Internal memory may include forms such as volatile memory, random access memory (RAM), and / or non-volatile memory within a computer-readable medium, such as read-only memory (ROM) and flash memory (flash RAM). Internal memory is an example of a computer-readable medium.

[0077] Computer-readable media, including permanent and non-permanent, movable and immovable media, can store information by any method or technique. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable and programmable read-only memory (EEPROM), flash memory or other internal memory technologies, read-only optical disc memory (CD-ROM), digital multi-purpose discs (DVDs) or other optical memory, and magnetic tape cassettes. Magnetic tape, magnetic disk memory or other magnetic storage devices or any other non-transmission media can be used to store information accessible by computing devices. According to the classifications in this text, computer-readable media do not include transient computer-readable media such as modulated data signals or carrier waves.

[0078] It should be further clarified that the terms “encompassing,” “including,” or any other variations thereof, by covering non-exclusive inclusion, mean that a process, method, product, or device containing a set of elements not only contains those elements but also other elements not explicitly listed, or elements specific to that kind of process, method, product, or device. In less restrictive contexts, the element limited by the phrase “including one…” does not preclude the presence of yet another identical element within a process, method, product, or device containing the aforementioned element.

[0079] Those skilled in the art will know that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment that combines both software and hardware aspects. The present invention may also take the form of a computer program product implemented on one or more computer-compatible storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) containing computer-compatible program code.

[0080] The present invention can be described in the general context of computer-executable instructions, such as program modules, which are executed by computers. Generally, program modules include routines, programs, objects, assemblies, data structures, etc., that perform a specific task or realize a specific abstract data type. The present invention can also be put into practice in distributed computing environments, in which tasks are executed by remote processing units connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0081] Each embodiment of the present invention is described by a progressive manner, and identical or similar parts between embodiments can be referenced to one another. Each embodiment focuses on parts that differ from the other embodiments. In particular, the system embodiments are described relatively briefly because they are fundamentally similar to the method embodiments, so relevant parts can be found by referring to the partial description of the method embodiments.

[0082] The above describes only embodiments of the present invention and does not limit it. Those skilled in the art can modify and alter the present invention in various ways. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention must be included within the claims of the present invention.

Claims

1. A three-dimensional target detection method based on the votenet model, comprising: Step 1: constructing the votenet model; Step 2 involves constructing a point cloud dataset for training the votetnet model, tailored to the target of interest. Step 3 involves constructing a seed point displacement loss function based on a two-layer nested three-dimensional rectangular box spatial partitioning for training the votenet model, which refers to the degree to which a seed point located in the background region near the target moves to the vicinity of the target's central region during the displacement phase, in order to calculate the seed point displacement loss, and Step 4 involves constructing additional loss functions for training the votet model, including foreground-background classification loss function, center offset loss function, dimension offset loss function, and direction angle offset loss function. Step 5 involves training the votenet model based on the constructed point cloud dataset and loss function. Step 6 involves using an RGB-D camera to acquire point cloud data of the scene to be detected, Step 7 outputs the 3D target detection result of the target of interest through the votet model based on the point cloud data of the scene to be detected, A three-dimensional target detection method based on the votenet model, including the following.

2. The three-dimensional target detection method based on the votenet model according to claim 1, characterized in that in step 2, the point cloud dataset refers to a plurality of point cloud data samples including a target of interest and annotation information of the three-dimensional rectangular bounding box of each target of interest in the sample.

3. A three-dimensional target detection method based on the votenet model according to claim 2, characterized in that the point cloud data may be from multiple types of sensors.

4. In step 3 above, for each training sample, the seed point displacement loss function is: Substep (3.1) involves inputting the point cloud data of the training sample into the votenet model and obtaining the seed point coordinates extracted by the votenet model and the coordinate offset of each seed point predicted by the votenet model. Assuming the training sample contains a total of n targets, for the i ∈ [1, nt]th target, the actual three-dimensional rectangular bounding box of the target is defined as Boxi = (xi, yi, zi, li, wi, hi, θi), where xi, yi, zi are the center point coordinates of the bounding box, li, wi, hi are the length, width, and height of the bounding box, and θi is the direction angle of the bounding box. Substep (3.2) generates a three-dimensional rectangular box Boxi(e) = (xi, yi, zi, k*li, k*wi, k*hi, θi) for each actual three-dimensional rectangular bounding box Boxi of the target, where the center point coordinates and direction angle of the bounding box are the same, but the length, width, and height are each enlarged by k times. Let ns be the total number of seed points, and traverse all seed points, so j ∈ [1, n For the s]th seed point Pj, let the coordinates of seed point Pj in three-dimensional space be (xj, yj, zj), and traverse the real three-dimensional rectangular bounding box Boxi, i ∈ [1, nt] of the entire target in accordance with seed point Pj, and if the coordinates (xj, yj, zj) of seed point Pj lie inside or on the outer surface of a real three-dimensional rectangular bounding box Boxi, a corresponding target point Tj is generated for that seed point, and the coordinate value of the target point Tj is equal to the center point coordinate value (xi, yi, zi) of the real three-dimensional rectangular bounding box Boxi, and if the coordinates (xj, yj, zj) of seed point Pj lie inside or on the outer surface of multiple real three-dimensional rectangular bounding boxes, the coordinate value of the target point Tj is equal to the center point coordinate of the three-dimensional rectangular bounding box closest to seed point Pj among the center points of the real three-dimensional rectangular bounding boxes in substep (3.3), Let n's be the number of seed points without corresponding target points. Traverse all seed points without corresponding target points. For the j ∈ [1, n's]th seed point Pj, traverse the expanded three-dimensional rectangular bounding box Boxi(e), i ∈ [1, nt] to match seed point Pj. If the coordinates (xj, yj, zj) of seed point Pj lie inside or on the outer surface of the expanded three-dimensional rectangular bounding box Boxi(e), generate a corresponding target point Vj for that seed point, and the coordinates of the target point Vj are equal to the coordinates (xj, yj, zj) of seed point Pj in substep (3.4). Substep (3.5) involves traversing the entire seed points, and for the j ∈ [1, ns]th seed point Pj, the votetet model predicts a coordinate offset for that seed point as (Δxj, Δyj, Δzj), and if a target point Tj or target point Vj exists corresponding to seed point Pj, a corresponding predicted point Qj is generated for seed point Pj, and the coordinate values ​​of the predicted point Qj are (xj + Δxj, yj + Δyj, zj + Δzj), A substep for calculating the seed point displacement loss Lvote-reg, where PTk is the seed point Pk where the target point Tk exists, nT is the total number of PTk, PVm is the seed point Pm where the target point Vm exists, nV is the total number of PVm, D(A,B) is the distance between points A and B in three-dimensional space, and α is the equilibrium factor, then the specific formula for calculating the seed point displacement loss is: Substep (3.6) is as follows: A three-dimensional target detection method based on the votenet model according to claim 1, characterized in that it is realized by the above.

5. The method for detecting a three-dimensional target based on the votenet model according to claim 1, characterized in that, in step 4, the other loss functions used for training the votenet model refer to the foreground-background classification loss function, the center offset loss function, the dimension offset loss function, and the direction angle offset loss function of the votenet model.

6. A three-dimensional target detection method based on a votenet model according to claim 1, characterized in that the training described in step 5 refers to obtaining optimized votenet model parameters using a backpropagation algorithm.

7. A three-dimensional target detection method based on the votenet model according to claim 1, characterized in that the point cloud data described in step 6 refers to point cloud data generated by projecting the pixels of the depth image generated by the RGB-D camera into three-dimensional space based on the internal parameters of the RGB-D camera.

8. A votenet model construction module for building a votenet model, A point cloud dataset construction module for building point cloud datasets to train votet models according to the target of interest, To calculate the seed point displacement loss, a seed point displacement loss function construction module is used to construct a seed point displacement loss function based on a two-layer nested three-dimensional rectangular box spatial partitioning for training the votenet model. This function refers to the degree to which a seed point located in the background region near the target moves to the vicinity of the target's central region during the displacement phase. An additional loss function construction module for building other loss functions for training the votet model, including foreground-background classification loss function, center offset loss function, dimension offset loss function, and direction angle offset loss function. A votenet model building module for training a votenet model based on a constructed point cloud dataset and loss function, A point cloud data acquisition module for the scene to be detected, which uses an RGB-D camera to acquire point cloud data of the scene to be detected, A result output module for outputting 3D target detection results of the target of interest through the votenet model, based on point cloud data of the scene to be detected. A three-dimensional target detection system based on the votenet model, characterized by including the following:

9. A computer-readable storage medium characterized in that it stores a program that, when executed by a processor, realizes a three-dimensional target detection method based on the votenet model described in any one of claims 1 to 7.

Images