Target detection method and device, electronic equipment and storage medium thereof

Abstract

The invention discloses a target-based detection method and device, electronic equipment and a storage medium thereof, and relates to the field of artificial intelligence. The method comprises the steps of obtaining original point cloud data of an environment where a first object is located; performing point cloud completion on the original point cloud data to determine completed point cloud data; inputting the complemented point cloud data into a pre-trained first detection network, and outputting an initial candidate box and a target coding value of the first object; inputting the fusion feature of the complemented point cloud data, the target coding value and the initial candidate box into a pre-trained second detection network, and outputting a target candidate box of the first object; network layers included in the first detection network and the second detection network are different; the detection result of the first object is determined according to the target candidate box, the problem of inaccurate detection of the first object caused by the original point cloud data can be avoided, the target candidate box is determined by combining the second detection network with the first detection network, and the detection accuracy of the first object is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and more particularly, to a target detection method and device, an electronic device, and a storage medium thereof. BACKGROUND

[0002] With the rapid development of technology, artificial intelligence is increasingly widely used in people's daily life, and object detection has become one of the important cores of the application of artificial intelligence in different tasks. At present, objects can be detected by collected images or by point cloud data. However, the precision of object detection from collected images is low, and the detection of objects from point cloud data can result in a lack of point cloud data due to different factors in the collection process, thereby failing to accurately detect the corresponding target object. Therefore, how to improve the detection accuracy of target objects has become a problem to be solved. SUMMARY

[0003] In view of the above problems, the embodiments of the present application provide a target detection method and device, and an electronic device to improve the above problems.

[0004] According to a first aspect of the embodiments of the present application, a target detection method is provided, which includes: acquiring original point cloud data of an environment where a first object is located; performing point cloud completion on the original point cloud data to determine completed point cloud data; inputting the completed point cloud data into a pre-trained first detection network to output an initial candidate box and a target encoding value of the first object, wherein the target encoding is used to indicate an orientation angle of the object in the environment, and the initial candidate box is used to frame a two-dimensional region of the first object in the environment; inputting fusion features of the completed point cloud data, the target encoding value, and the initial candidate box into a pre-trained second detection network to output a target candidate box of the first object, wherein the first detection network and the second detection network contain different network layers, and the target candidate box is used to frame a three-dimensional region of the first object in the environment, which includes the orientation angle of the first object in the environment; and determining a detection result of the first object according to the target candidate box.

[0005] In some embodiments, the inputting of the completed point cloud data into the pre-trained first detection network to output the initial candidate box and the target encoding value of the first object includes: performing orientation angle detection on the completed point cloud data by the first detection network to determine the orientation angle of the first object in the environment; and determining the target encoding value according to the orientation angle and a mapping relationship, wherein the mapping relationship is used to indicate a respective encoding value corresponding to each of a plurality of orientation angles.

[0006] In some embodiments, the determining the target encoding value according to the mapping relationship of the orientation angle comprises: determining a residual angle corresponding to the orientation angle and an angle width of the orientation angle according to the mapping relationship; and determining the target encoding value according to the angle width, the residual angle and the orientation angle.

[0007] In some embodiments, the point cloud completion on the original point cloud data to determine the completed point cloud data comprises: matching the original point cloud data with each sample point cloud set in a sample point cloud library, determining a similarity between the original point cloud data and each sample point cloud set in the sample point cloud library; determining a target sample point cloud according to the similarity, and performing point cloud completion on a region in the original point cloud data that lacks point cloud data compared with the target sample point cloud based on the target sample point cloud to obtain the completed point cloud data.

[0008] In some embodiments, the matching the original point cloud data with each sample point cloud set in the sample point cloud library to determine the similarity between the original point cloud data and each sample point cloud set in the sample point cloud library comprises: determining a first similarity score according to first coordinate information of the original point cloud data and second coordinate information of each sample point cloud set in the sample point cloud library; determining a first bounding box corresponding to the original point cloud data and a second bounding box of each sample point cloud set in the sample point cloud library, and determining a second similarity score between the first bounding box and the second bounding box; determining an additional voxel number of each sample point cloud set in the sample point cloud library compared with the original point cloud data; and determining the similarity according to the first similarity score, the second similarity score and the additional voxel number.

[0009] In some embodiments, the inputting the completed point cloud data into the pre-trained first detection network to output the initial candidate box of the first object and the target encoding value comprises: determining a confidence corresponding to each point cloud in the completed point cloud data; determining point cloud data corresponding to a confidence greater than a confidence threshold value as target detection point cloud data; and inputting the target detection point cloud data into the pre-trained first detection network to perform initial detection on a region where the first object is located, and outputting the initial candidate box of the first object.

[0010] In some embodiments, before the fusion feature of the completed point cloud data, the target encoding value, and the initial candidate box are input into the pre-trained second detection network to output the target candidate box of the first object, the method further comprises: performing feature extraction on the completed point cloud data to determine local geometric features and neighborhood topological features corresponding to the completed point cloud data; performing voxel processing on the completed point cloud data to determine voxel occupancy features corresponding to the completed point cloud data; and performing feature fusion on the voxel occupancy features, the local geometric features, and the neighborhood topological features to obtain the fusion feature.

[0011] According to a second aspect of the embodiments of the present application, a target detection device is provided, which comprises: an original point cloud data acquisition module configured to acquire original point cloud data of an environment in which a first object is located; a point cloud completion module configured to perform point cloud completion on the original point cloud data to determine completed point cloud data; an initial candidate box determination module configured to input the completed point cloud data into a pre-trained first detection network to output an initial candidate box and a target encoding value of the first object, wherein the target encoding is used to indicate an orientation angle of the object in the environment, and the initial candidate box is used to frame a two-dimensional region of the first object in the environment; a target candidate box determination module configured to input a fusion feature of the completed point cloud data, the target encoding value, and the initial candidate box into a pre-trained second detection network to output a target candidate box of the first object, wherein the first detection network and the second detection network comprise different network layers, and the target candidate box is used to frame a three-dimensional region of the first object in the environment, which includes the orientation angle of the first object in the environment; and a detection result determination module configured to determine a detection result of the first object according to the target candidate box.

[0012] According to a third aspect of the embodiments of the present application, an electronic device is provided, which comprises: a processor; and a memory having computer readable instructions stored thereon, wherein the computer readable instructions, when executed by the processor, implement the target detection method as described above.

[0013] According to a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, having computer readable instructions stored thereon, wherein the computer readable instructions, when executed by a processor, implement the target detection method as described above.

[0014] In the scheme of the present application, first, the original point cloud data of the environment where the first object is located is subjected to point cloud completion to obtain completed point cloud data, and then the completed point cloud data is input into the first detection network to output an initial candidate box for framing a two-dimensional region of the first object in the environment and a target encoding value indicating the orientation angle of the object in the environment, and then the fusion features of the initial candidate box, the target encoding value and the completed point cloud data are input into a second detection network different from the network result of the first detection network to obtain a target candidate box for framing a three-dimensional region of the first object in the environment, and finally the detection result of the first object can be determined based on the determined target candidate box. By completing the original point cloud data, the accuracy of the determination of the candidate box in the first stage by the first detection network based on the completed point cloud data is ensured, and the problem of inaccurate detection of the first object due to the sparseness of the original point cloud data or the incompleteness of the point cloud data corresponding to the first object caused by occlusion is avoided. And the fusion features of the output of the first detection network and the completed point cloud data are subjected to the determination of the candidate box in the second stage by the second detection network again, ensuring the accuracy of the target candidate box for framing the three-dimensional region of the first object in the environment, and further improving the detection accuracy of the first object.

[0015] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained from these drawings without creative labor for those skilled in the art.

[0017] Figure 1 is a system schematic diagram for determining the candidate box of a three-dimensional object according to an embodiment of the present application.

[0018] Figure 2 is a flowchart of a target detection method according to an embodiment of the present application.

[0019] Figure 3 is a flowchart of a target detection method according to another embodiment of the present application.

[0020] Figure 4 is a flowchart of a target detection method according to still another embodiment of the present application.

[0021] Figure 5This is a flowchart illustrating the specific steps of step 420 according to an embodiment of this application.

[0022] Figure 6 This is a comparative schematic diagram illustrating point cloud completion according to an embodiment of this application.

[0023] Figure 7 This is a flowchart illustrating a target detection method according to yet another embodiment of this application.

[0024] Figure 8 This is a flowchart illustrating a target detection method according to another embodiment of this application.

[0025] Figure 9 This is a block diagram of a target detection device according to an embodiment of this application.

[0026] Figure 10 This is a hardware structure diagram of an electronic device according to another embodiment of this application.

[0027] The accompanying drawings have illustrated specific embodiments of the present invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the inventive concept in any way, but rather to illustrate the concept of the invention to those skilled in the art through specific embodiments. Detailed Implementation

[0028] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0029] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0030] Please see Figure 1 , Figure 1 A system for determining candidate bounding boxes of a three-dimensional object is shown according to an embodiment of this application.

[0031] The following provides an exemplary description of the specific implementation methods of each module included in system 100.

[0032] In an optional implementation, the system 100 comprises an upper controller module and a lower controller module, where the upper controller module and the lower controller module refer to software units or modules. The upper controller module is configured to determine the control instruction of the electronic device, and the lower controller module is configured to determine the candidate box of the three-dimensional object.

[0033] In another optional implementation, the upper controller module and the lower controller module refer to hardware devices.

[0034] For example, the upper controller module can include, but is not limited to, a vehicle-mounted central processor.

[0035] For another example, the lower controller module can include, but is not limited to, a point cloud completion module 110, a voxel feature determination module 120, a region candidate module 130, a backbone feature determination module 140, a feature fusion module 150, and a candidate box refinement module 160. The point cloud completion module 110 is configured to complete the original point cloud data of the environment where the first object is located to obtain completed point cloud data. The voxel feature determination module 120 is configured to determine the voxel occupancy feature of the completed point cloud data output by the point cloud completion module 110. The region candidate module 130 is configured to output the initial candidate box of the first object and the target encoding value based on the completed point cloud data output by the point cloud completion module 110 as input. The backbone feature determination module 140 is configured to determine the backbone feature of the completed point cloud data output by the point cloud completion module 110. The feature fusion module 150 is configured to fuse the voxel occupancy feature output by the voxel feature determination module 120 and the backbone feature output by the backbone feature determination module 140 to obtain the fusion feature of the completed point cloud data. The candidate box refinement module 160 is configured to output the target candidate box of the first object based on the fusion feature output by the feature fusion module 150 and the initial candidate box and the target encoding value output by the region candidate module 130 as input. In this way, the first object can be detected based on the target candidate box, so that the vehicle-mounted refrigerator can control the temperature to be set in the case of failure of the temperature sensor.

[0036] Optionally, the upper controller module and the lower controller module can communicate through wired connection or wireless connection. For example, the wired connection can include, but is not limited to, bus, optical fiber, network cable, etc. The wireless connection includes transmission control protocol / internet protocol (TCP / IP), network protocol for realizing remote direct memory access (RDMA) through Ethernet (RDMA over converged Ethernet, RoCE), etc.

[0037] The upper controller module and the lower controller module shown below Figure 1 The vehicle control method provided by the embodiments of the present application is exemplarily described.

[0038] Please refer to Figure 2 , Figure 2 The target detection method provided by an embodiment of the present application is shown. In a specific embodiment, the target detection method can be applied to a target detection device 700 as shown in Figure 9 and an electronic device 800 configured with the target detection device 700. Figure 10 The specific process of the embodiment will be described below. It can be understood that the method can be executed by an electronic device with computing processing capability, such as a desktop computer, a notebook computer, a vehicle terminal, a vehicle processor, and the like. The target detection method will be described in detail with reference to the flow shown in Figure 2 The target detection method can specifically include the following steps:

[0039] Step 210: Obtain original point cloud data of an environment where a first object is located.

[0040] As an optional way, the original point cloud data of the environment where the first object is located can be collected by a radar sensor. The radar sensor can be a sensor arranged on the vehicle body. In this way, the objects located around the vehicle can be detected by the original point cloud data collected by the radar sensor. In this way, the vehicle can avoid the object that is about to collide, so as to ensure the driving safety of the vehicle.

[0041] Optionally, the first object can be any object in the environment where the vehicle is located and within the detection range of the sensor of the vehicle, or any object in the set target area. The first object is not specifically limited here.

[0042] Step 220: Perform point cloud completion on the original point cloud data to determine completed point cloud data.

[0043] As an optional way, when the point cloud data of the first object is collected, the first object can be blocked, so that the original point cloud data cannot completely represent the first object. In addition, the original point cloud data of the first object can be incomplete due to the sensor itself. If the first object is directly detected according to the original point cloud data, the detection result of the first object can not be accurate, and even the detection error of the first object can occur. Therefore, in order to ensure the detection accuracy of the first object, after the original point cloud data of the first object is obtained, the original point cloud data can be completed to obtain completed point cloud data that can accurately represent the first object.

[0044] In step 230, the completed point cloud data is input into the pre-trained first detection network to output an initial candidate box of the first object and a target encoding value, wherein the target encoding is used to indicate the orientation angle of the object in the environment, and the initial candidate box is used to frame the two-dimensional region of the first object in the environment.

[0045] As an optional mode, since the detection accuracy of the first object by the image data depends on the accuracy of the image data, and the light has a great influence on the image, and thus the accuracy of the image data is low in the case of complex light, which leads to very low detection accuracy of the first object, and the image data is in two-dimensional space, while the first object is in three-dimensional space, in order to detect the first object, the two-dimensional image data needs to be projected into three-dimensional space, which leads to low detection accuracy of the first object. However, the point cloud data naturally exists in three-dimensional space and is not affected by the light of the environment, and does not need to be converted between two-dimensional space and three-dimensional space, therefore, detecting the first object by the point cloud data in three-dimensional space can ensure the detection accuracy of the first object.

[0046] Optionally, since each object has its own orientation angle in three-dimensional space, in order to further improve the detection accuracy of the first object, in the process of determining the initial candidate box corresponding to the two-dimensional region of the first object in the environment by the first detection network according to the completed point cloud data, the target encoding value corresponding to the orientation angle of the first object in the environment is determined, so as to facilitate the orientation angle of the object in the environment as accurate input data for detecting the first object, thereby improving the detection accuracy of the first object.

[0047] Optionally, the mapping relationship between the orientation angle and the encoding value can be pre-set, so that after the orientation angle of the first object is determined, the mapping relationship can be queried to determine the target encoding value. Optionally, since the orientation angle cannot maintain its continuity when performing regression prediction, after the orientation angle of the first object in the environment is determined, the orientation angle can be encoded so that the orientation angle can be represented in the form of the encoding value, avoiding the discontinuity of the orientation angle when determining the candidate box corresponding to the first object, and further ensuring the accuracy of the target candidate box output by the second detection network.

[0048] Optionally, before using the first detection network, a plurality of sets of sample point cloud data can be pre-set, each set of sample point cloud data including a first actual candidate frame of a sample object corresponding to the sample point cloud data and an actual encoding value corresponding to an orientation angle of the sample object in the environment, and then the plurality of sets of sample point cloud data are input into the first detection network for training to output a first reference candidate frame and a reference encoding value, and then the first difference between the first reference candidate frame and the first actual candidate frame and the second difference between the reference encoding value and the actual encoding value are calculated by decibel to determine the first training loss value of the first detection network according to the first difference and the second difference, and if the first training loss value is greater than or equal to a first threshold, the parameters of the first detection network are adjusted, and the plurality of sets of sample point cloud data are trained again based on the first detection network after the parameter adjustment until the first training loss value is less than the first threshold.

[0049] Optionally, in order to ensure the detection accuracy of the first detection network, the actual candidate frame category corresponding to each set of sample point cloud data in the plurality of sets of sample point cloud data can be set, which is used to indicate whether the corresponding sample object is an object (which can be understood as foreground) or a non-object (which can be understood as background) in the environment, and when training the first detection network, the classification loss between the actual candidate frame category corresponding to each set of sample point cloud data in the plurality of sets of sample point cloud data and the reference candidate frame category determined by the first detection network and the first candidate frame loss between the first reference candidate frame and the first actual candidate frame can be determined respectively, so as to determine the first training loss value of the first detection network based on the classification loss and the first candidate frame loss, and the parameters of the first detection network are adjusted based on the size relationship between the first training loss value and the first threshold until the first training loss value is less than the first threshold. Wherein, the classification loss can be determined by the formula , wherein p t is used to indicate the category label of the corresponding sample point cloud data, and the sample category label is , wherein p p is the score of the sample being predicted as a non-object, and L focal (·) is the focal loss, and α and γ are hyperparameters, which can be set as α = 0.25 and γ = 2. The first candidate frame loss can be determined by the formula , wherein a is used to indicate the point cloud coordinates of the first candidate frame, the length, width and height of the candidate frame.

[0050] In step 240, the fusion feature of the completed point cloud data, the target encoding value, and the initial candidate box are input into a pre-trained second detection network, and a target candidate box of the first object is output, wherein the first detection network and the second detection network contain different network layers, and the target candidate box is used to frame a three-dimensional region of the first object in the environment, and the three-dimensional region includes the orientation angle of the first object in the environment.

[0051] As an optional mode, before the target candidate box for framing the three-dimensional region of the first object is further determined by the second detection network, the fusion feature of the completed point cloud data can be determined first, so that the second detection network determines the target candidate box through multi-source input. Optionally, the local geometric feature and the neighborhood topological feature corresponding to the completed point cloud data can be extracted by a pre-trained feature extraction network, so that feature fusion can be performed based on the local geometric feature and the neighborhood topological feature, and the fusion feature is obtained.

[0052] Optionally, since the first detection network cannot directly fuse the obtained orientation angle of the first object in the three-dimensional space into the corresponding initial candidate box, the region framed by the initial candidate box does not include the orientation angle, and the region framed by the initial candidate box cannot accurately match the actual position of the first object in the three-dimensional space, so that the detection of the first object is not accurate enough. Therefore, the initial candidate box output by the first detection network is refined by the second detection network using the target encoding value representing the orientation angle of the first object and the fusion feature representing the first object, so that the target candidate box is obtained.

[0053] Optionally, before the target candidate box is obtained by the second detection network, a plurality of groups of sample candidate boxes can be set, each group of sample candidate boxes corresponds to an actual candidate box, and each group of sample candidate boxes is provided with a corresponding reference encoding value and a reference fusion feature. In this way, the second detection network outputs a predicted candidate box according to the plurality of groups of sample candidate boxes and the reference encoding value and the reference fusion feature corresponding to each group of candidate boxes in the plurality of groups of sample candidate boxes. Then, a third difference value is determined by the predicted candidate box and the actual candidate box. In this way, the third difference value is used as a second training loss value of the second detection network. If the second training loss value is greater than or equal to a second threshold value, the parameters of the second detection network are adjusted, and the second detection network after the parameter adjustment is used to train again based on the plurality of groups of sample candidate boxes and the reference encoding value and the reference fusion feature corresponding to each group of candidate boxes in the plurality of groups of sample candidate boxes, until the second training loss value is less than the second threshold value, and the training of the second detection network is completed.

[0054] Optionally, a third difference value between the predicted bounding box and the actual bounding box can be determined by calculating a center position loss, a scale loss and a direction angle loss corresponding to the predicted bounding box and the actual bounding box, and performing weighted fusion according to the center position loss, the scale loss and the direction angle loss. The third difference value can be represented by the formula L reg =w ATE ×L trans +w ASE ×L scale +w AOE ×L orient , wherein L trans is the center position loss, L scale is the scale loss, L orient is the direction angle loss, w ATE is a first weight of the center position loss, w ASE is a second weight of the scale loss, and w AOE is a third weight of the direction angle. Optionally, during the training process of the second detection network, the first weight, the second weight and the third weight w tpe ={w ATE ,w ASE ,w AOE} can be set to an initial value, and then during each training process of the network, the first weight, the second weight and the third weight can be updated according to three average errors TPE = {ATE, ASE, AOE} of the center position loss, the scale loss and the direction angle loss, and each time according to the three average errors. The first weight, the second weight and the third weight can be updated by the formula , wherein i is the number of training of the second detection network.

[0055] In step 250, a detection result of the first object is determined according to the target bounding box.

[0056] As an optional way, after the target bounding box is determined, since the target bounding box frames the region corresponding to the first object in the three-dimensional space of the environment, after the target bounding box is determined, the target point cloud data corresponding to the target bounding box can be grouped based on a clustering algorithm to obtain different groups of point cloud data based on distance or density, and finally the different groups of point cloud data can be identified based on a gridding method to convert the different groups of point cloud data into triangular patches or voxel models, so as to determine the first object and obtain the detection result of the first object. The gridding method can include triangulation (such as Poisson reconstruction, spherical wave transform) and voxelization (such as voxel gridding, octree gridding).

[0057] In the embodiments of this application, the original point cloud data of the environment where the first object is located is first completed by point cloud completion to obtain completed point cloud data. Then, the completed point cloud data is input into the first detection network, which outputs an initial candidate box for selecting the two-dimensional region of the first object in the environment and a target encoding value indicating the orientation angle of the object in the environment. Then, the fusion features of the initial candidate box, the target encoding value and the completed point cloud data are input as input data to a second detection network that is different from the network result of the first detection network to obtain a target candidate box for selecting the three-dimensional region of the first object in the environment. Finally, the detection result of the first object can be determined based on the determined target candidate box. By completing the original point cloud data, the accuracy of the first detection network in determining the candidate boxes in the first stage is ensured. This avoids the problem of inaccurate detection of the first object due to the sparsity of the original point cloud data or the incomplete point cloud data corresponding to the occluded first object. Furthermore, the second detection network uses the fusion features of the output of the first detection network and the completed point cloud data to determine the candidate boxes in the second stage, ensuring the accuracy of the target candidate boxes used to select the three-dimensional region of the first object in the environment, and further improving the detection accuracy of the first object.

[0058] For the process of inputting the completed point cloud data into the pre-trained first detection network and outputting the initial candidate bounding box and target encoding value of the first object, please refer to [link to relevant documentation]. Figure 3 , Figure 3 This illustration shows a target detection method provided by an embodiment of this application. The following will focus on... Figure 3 The process shown will be described in detail. The target detection method may specifically include the following steps:

[0059] Step 310: Obtain the original point cloud data of the environment where the first object is located.

[0060] Step 320: Perform point cloud completion on the original point cloud data to determine the completed point cloud data.

[0061] Step 330: The orientation angle of the completed point cloud data is detected by the first detection network to determine the orientation angle of the first object in the environment and the initial candidate box of the first object.

[0062] As an optional mode, since the point cloud data contains corresponding three-dimensional coordinates in the three-dimensional world, the orientation angle of the first object in the environment can be obtained by detecting the orientation angle of each point cloud in the completed point cloud data through the first detection network. Optionally, the orientation angle of the first object can be the corresponding angle of the first object relative to the device provided with the radar point cloud sensor, or the yaw angle of the first object, for example, a car is north, east, or forms a corresponding angle with the reference object (for example, a road).

[0063] In step 340, the target encoding value is determined according to the orientation angle and a mapping relationship, wherein the mapping relationship is used to indicate the respective encoding value corresponding to each of the plurality of orientation angles.

[0064] As an optional mode, the mapping relationship between different angles and corresponding encoding values can be set in advance, so that after the orientation angle of the first object in the environment is determined, the target encoding value corresponding to the orientation angle of the first object in the environment can be determined by querying the mapping relationship.

[0065] Optionally, the maximum angle interval in the environment can be scored based on the set fixed size of the angle, so as to obtain a plurality of small intervals, the angle difference in each small interval being the same, and then the respective angle range of each small interval is classified and encoded, so as to obtain a plurality of encoding values corresponding to the respective small intervals, and finally, the mapping relationship is obtained by associating different angles with encoding values.

[0066] In some embodiments, the step 340 comprises: determining a residual angle corresponding to the orientation angle and an angle width of the orientation angle according to the mapping relationship; and determining the target encoding value according to the angle width, the residual angle and the orientation angle.

[0067] As an optional mode, in order to ensure that the encoding value can accurately represent the corresponding orientation angle, the residual angle of the orientation angle and the angle width corresponding to each encoding value in the mapping relationship can be determined based on the mapping relationship. Since there is a certain angle difference in each interval obtained by equally dividing the maximum angle (for example, 360°) of the three-dimensional region where the first object is located in the mapping relationship, the angle difference is the corresponding angle width. Optionally, the encoding category corresponding to the orientation angle of the first object can be determined based on the mapping relationship, and then the angle range corresponding to the encoding category is determined, so as to determine the angle difference based on the angle range, so as to obtain the angle width. The residual angle can be determined by calculating the difference between the encoding category of the orientation angle in the mapping relationship and the orientation angle.

[0068] Optionally, since the orientation angle of the first object is continuous, but the second detection network determines the target bounding box based on the orientation angle, the continuous orientation angle is discretized, which increases the error of the orientation angle. Therefore, before the orientation angle is input into the second detection network, the orientation angle is first continuously angle-rotated encoded to obtain a target encoding value that can be discretized, thereby avoiding the discretization error of the orientation angle and avoiding the angle boundary jitter problem, and improving the smoothness and stability of the determination of the target encoding value. Optionally, the target encoding value can be angle-encoded according to the angle width and the residual angle obtained by querying the mapping relationship according to the orientation angle. Optionally, the target encoding value can be determined by the formula , wherein, is the angle width corresponding to the orientation angle; is the encoding category corresponding to the orientation angle, and is the residual angle corresponding to the orientation angle, and the orientation angle is continuously angle-rotated encoded by a sine function and a cosine function to obtain the target encoding value.

[0069] Step 350: input the fusion feature of the completed point cloud data, the target encoding value, and the initial bounding box into a pre-trained second detection network, and output a target bounding box of the first object, wherein the network layers included in the first detection network and the second detection network are different, and the target bounding box is used to frame a three-dimensional region of the first object in the environment, and the three-dimensional region includes the orientation angle of the first object in the environment.

[0070] Step 360: determine a detection result of the first object according to the target bounding box.

[0071] The specific steps of steps 310-320 and steps 350-360 can be described in steps 210-220 and steps 240-250, and will not be described here.

[0072] In this embodiment, the first detection network determines the orientation angle of the first object in the environment according to the completed point cloud data, and determines the target encoding value of the orientation angle by querying the mapping relationship, so as to ensure that the second detection network can increase the orientation angle of the first object in the environment to further improve the accuracy of the target bounding box, thereby improving the detection accuracy of the first object.

[0073] For the process of point cloud completion of the original point cloud data and determination of the completed point cloud data, please refer to Figure 4 , Figure 4 The target detection method provided by an embodiment of the present application is shown. In the following, Figure 4The flowchart is illustrated in detail, and the target detection method can specifically include the following steps:

[0074] In step 410, original point cloud data of an environment where the first object is located is acquired.

[0075] In step 420, the original point cloud data is matched with each sample point cloud set in the sample point cloud library to determine the similarity between the original point cloud data and each sample point cloud set in the sample point cloud library.

[0076] As an optional mode, since the original point cloud data may be incomplete due to the first object being blocked by other objects or the self factors of the radar point cloud sensor during the acquisition process, and thus the accuracy of detecting the first object based on the original point cloud data is low, the original point cloud data can be used for point cloud completion to ensure that the completed point cloud data is used to detect the first object.

[0077] Optionally, the sample point cloud library can be pre-set, and a plurality of different objects and a plurality of different object corresponding sample point cloud sets are included in the sample point cloud library, so that the similarity between each sample point cloud set in the sample point cloud library and the original point cloud data can be determined by matching the original point cloud data with each sample point cloud set in the sample point cloud library, so that the sample point cloud set with the highest similarity is determined as the sample point cloud set closest to the original point cloud data corresponding to the first object, and thus the original point cloud data can be completed based on the sample point cloud set.

[0078] Optionally, the contour similarity between the original point cloud data and each sample point cloud set in the sample point cloud library can be calculated as the similarity between the original point cloud data and each sample point cloud set in the sample point cloud library, and other methods can also be used to determine the similarity between the original point cloud data and each sample point cloud set in the sample point cloud library.

[0079] In some embodiments, as shown in the figure, Figure 5 The step 420 includes the following steps 421 to 424.

[0080] In step 421, a first similarity score is determined according to the first coordinate information of the original point cloud data and the second coordinate information of each sample point cloud set in the sample point cloud library.

[0081] As an optional mode, the matching degree between the original point cloud data and each sample point cloud set in the sample point cloud library can be measured by the method of taking the inverse of the semi-chamfer distance formula, so that the first similarity score is calculated according to the first coordinate information of the original point cloud data and the second coordinate information of each sample point cloud set in the sample point cloud library. Optionally, the formula to determine a first similarity score, wherein p A is the original point cloud data, p B is each sample point cloud set in the sample point cloud library, and x and y represent values in coordinate information located on different coordinate axes, respectively.

[0082] Step 422, determining a first bounding box corresponding to the original point cloud data and a second bounding box of each sample point cloud set in the sample point cloud library, and determining a second similarity score between the first bounding box and the second bounding box.

[0083] As an optional way, in order to accurately complete the original point cloud data, it is necessary to determine a sample point cloud set similar to the contour of the first object, so as to complete the point cloud of the original point cloud data based on the sample point cloud set similar to the contour. In order to determine the sample point cloud set similar to the contour of the first object, the first bounding box of the original point cloud data and the second bounding box of each sample point cloud set in the sample point cloud library can be determined first.

[0084] Optionally, the edge point cloud in the original point cloud data can be determined first, so as to determine the first bounding box corresponding to the original point cloud data based on the edge point cloud, wherein the edge point cloud can be a point cloud group far away from the median point cloud of the original point cloud data. Optionally, each sample point cloud set in the sample point cloud library includes a respective bounding box, which can be obtained directly.

[0085] Optionally, the second similarity score can be determined by the Intersection over Union (IoU) of the first bounding box of the original point cloud data and the second bounding box of each sample point cloud set in the sample point cloud library after the center position and the orientation are aligned. Optionally, the second similarity score can be determined by the formula S2=αIoU(G A ,G B ), wherein G A is the first bounding box, G B is the second bounding box, and α is a coefficient.

[0086] Step 423, determining the number of additional voxels of each sample point cloud set in the sample point cloud library compared to the original point cloud data.

[0087] As an optional mode, in order to accurately supplement the original point cloud data, the number of additional voxels that the corresponding sample point cloud set in the sample point cloud library can provide to the original point cloud data can be determined, so that the data after point cloud completion is more complete. Optionally, the original point cloud data can be voxelized first to obtain a first voxel set corresponding to the original point cloud data, and each sample point cloud set in the sample point cloud library can be voxelized to obtain a corresponding second voxel set, and then the number of additional voxels of each sample point cloud set in the sample point cloud library compared with the original point cloud data is determined according to the first voxel set and the second voxel set.

[0088] Optionally, the number of additional voxels of each sample point cloud set in the sample point cloud library compared with the original point cloud data can be determined by the formula , wherein V A is the first voxel set corresponding to the original point cloud data, V B is the second voxel set corresponding to each sample point cloud set in the sample point cloud library.

[0089] Step 424, determining the similarity according to the first similarity score, the second similarity score and the number of additional voxels.

[0090] As an optional mode, the similarity between the original point cloud data and each sample point cloud set in the sample point cloud library can be obtained by adding the first similarity score, the second similarity score and the number of additional voxels. Optionally, the similarity between the original point cloud data and each sample point cloud set in the sample point cloud library can also be obtained by weighted fusion of the first similarity score, the second similarity score and the number of additional voxels, and the weight of the weighted fusion can be set according to actual needs, which is not limited herein.

[0091] Please continue to refer to Figure 4 Step 430, determining a target sample point cloud according to the similarity, and performing point cloud completion on the region in the original point cloud data that lacks point cloud data compared with the target sample point cloud based on the target sample point cloud to obtain the completed point cloud data.

[0092] As an optional mode, the sample point cloud set with the highest similarity in the sample point cloud library can be determined as the target sample point cloud, so that the original point cloud data can be completed by the target sample point cloud.

[0093] Optionally, after the target sample point cloud is determined, the point cloud sparse region of the original point cloud data compared with the target sample point cloud can be determined first, and the target point cloud data corresponding to the point cloud sparse region in the target sample point cloud can be determined, so that the point cloud completion of the original point cloud data can be realized by filling the target point cloud data in the corresponding sparse region. Figure 6As shown, the original point cloud data corresponding to the occluded vehicle and pedestrian is respectively completed to obtain relatively complete point cloud data.

[0094] Step 440, input the completed point cloud data into a pre-trained first detection network to output an initial candidate box of the first object and a target encoding value, wherein the target encoding is used to indicate the orientation angle of the object in the environment, and the initial candidate box is used to frame the two-dimensional region of the first object in the environment.

[0095] Step 450, input the fusion features of the completed point cloud data, the target encoding value, and the initial candidate box into a pre-trained second detection network to output a target candidate box of the first object, wherein the first detection network and the second detection network contain different network layers, the target candidate box is used to frame the three-dimensional region of the first object in the environment, and the three-dimensional region includes the orientation angle of the first object in the environment.

[0096] Step 460, determine the detection result of the first object according to the target candidate box.

[0097] The specific steps of steps 410 and steps 440-460 can be involved in steps 210 and steps 240-250, which will not be described here.

[0098] In this embodiment, the original point cloud data is matched with the sample point cloud set in the sample point cloud library to complete the point cloud data, which ensures the integrity of the completed point cloud data and avoids missing detection and false detection, thereby ensuring the detection accuracy of the first object. Moreover, the similarity between the original point cloud data and each sample point cloud set in the sample point cloud library is determined by calculating the first similarity score, the second similarity score, and the number of additional voxels of the original point cloud data and each sample point cloud set in the sample point cloud library, which ensures the accuracy of the point cloud completion of the original point cloud data.

[0099] For the process of inputting the completed point cloud data into the pre-trained first detection network to output the initial candidate box and the target encoding value of the first object, please refer to Figure 7 , Figure 7 The target detection method provided by an embodiment of the present application is shown. The following will be described in detail with respect to the flow shown in Figure 7 The target detection method can specifically include the following steps:

[0100] Step 510, obtain the original point cloud data of the environment where the first object is located.

[0101] Step 520, point cloud completion is performed on the original point cloud data to determine the completed point cloud data.

[0102] Step 530, the confidence of each point cloud in the completed point cloud data is determined.

[0103] As an optional way, in order to reduce the calculation amount of determining the initial candidate box by completing the point cloud data, the confidence of the point cloud data corresponding to the first object in the completed point cloud data is determined first, so that the completed point cloud data can be filtered based on the confidence, so that the point cloud data not belonging to the first object (such as noise point cloud, other background objects and flying insects) is deleted, and subsequent detection is only performed on the point cloud data belonging to the first object.

[0104] Optionally, the confidence of different point clouds in the point cloud completion data can be quickly predicted by a pre-trained lightweight convolutional network, and then the completed point cloud data can be filtered based on the confidence.

[0105] Step 540, the point cloud data corresponding to the confidence greater than the confidence threshold is determined as the target detection point cloud data.

[0106] As an optional way, after determining the confidence of each point cloud, the point cloud data with a confidence greater than a confidence threshold is determined as the point cloud data corresponding to the first object, so as to determine the target detection point cloud data, and then the target detection point cloud data is input into the first detection network as the input of the first detection network to determine the initial candidate box of the first object. Optionally, the process of screening the target detection point cloud data can be represented as Wherein, F lr represents the low-resolution feature map corresponding to the completed point cloud data; f θ (·) is a lightweight convolutional network; σ(·) represents Sigmoid, which is used to obtain the confidence of the target existence τ is the confidence threshold; C is the target detection point cloud data (i.e. the point cloud set corresponding to the high-confidence target region center) filtered by the fast candidate guide module.

[0107] Step 550, the target detection point cloud data is input into the pre-trained first detection network to perform initial detection on the region where the first object is located, and the initial candidate box of the first object is output.

[0108] As an optional way, since the number of target detection point cloud data is less than the number of completed point cloud data, the target detection point cloud data is input into the pre-trained first detection network to perform initial detection on the region where the first object is located, which can effectively reduce redundant calculation and significantly improve inference speed and detection efficiency, thereby improving the detection efficiency of the first object.

[0109] Step 560, input the fusion features of the completed point cloud data, the target encoding value, and the initial candidate box into a pre-trained second detection network, and output a target candidate box of the first object, wherein the first detection network and the second detection network contain different network layers, and the target candidate box is used to frame a three-dimensional region of the first object in the environment, which includes the orientation angle of the first object in the environment.

[0110] Step 570, determine the detection result of the first object according to the target candidate box.

[0111] The specific steps of steps 510-520 and steps 560-570 can be described in steps 210-220 and steps 240-250, which will not be repeated here.

[0112] In this embodiment, the confidence of each point cloud in the completed point cloud data is first confirmed, so that the completed point cloud data can be filtered based on the confidence of each point cloud, so that the calculation amount of the initial candidate box determined by the first detection network based on the filtered completed point cloud data is reduced, the determination efficiency of the initial candidate box is improved, and the detection efficiency of the first object is further improved.

[0113] For the above process before the fusion features of the completed point cloud data, the target encoding value, and the initial candidate box are input into the pre-trained second detection network, and the target candidate box of the first object is output, please refer to Figure 8 , Figure 8 A target detection method provided by an embodiment of the application is shown. The following will be described in detail with reference to the flow shown in Figure 8 The target detection method can specifically include the following steps:

[0114] Step 610, obtain the original point cloud data of the environment where the first object is located.

[0115] Step 620, perform point cloud completion on the original point cloud data to determine the completed point cloud data.

[0116] Step 630, input the completed point cloud data into a pre-trained first detection network to output an initial candidate box of the first object and a target encoding value, wherein the target encoding is used to indicate the orientation angle of the object in the environment, and the initial candidate box is used to frame a two-dimensional region of the first object in the environment.

[0117] Step 640, perform feature extraction on the completed point cloud data to determine the local geometric features and neighborhood topological features corresponding to the completed point cloud data.

[0118] As an optional mode, the feature extraction of the completed point cloud data can be performed by a pre-trained convolutional network to obtain the neighborhood topological features of the local geometric features corresponding to the completed point cloud data. Optionally, the local geometric features and the neighborhood topological features can be obtained by using a point-graph joint encoding mode. First, the local geometric features are obtained by determining the normal vector features, the curvature of the local surface bending degree of the completed point cloud data, and the relative coordinates and distances of each point cloud and its neighborhood center, and the like, based on the position information of the completed point cloud data.

[0119] Optionally, after obtaining the local geometric features, the mutual relationship and overall structure between each point in the neighborhood and the adjacent points in the local geometric features are determined to obtain the neighborhood topological features.

[0120] Step 650, voxel processing is performed on the completed point cloud data to determine the voxel occupancy features corresponding to the completed point cloud data.

[0121] As an optional mode, since the point cloud data is irregular and unordered, the detection efficiency of the second detection network based on the completed point cloud data for detection can be reduced. Therefore, the completed point cloud data can be voxelized to convert the irregular completed point cloud data into a regular three-dimensional grid.

[0122] Optionally, the corresponding three-dimensional boundary can be determined based on the completed point cloud data, and then the three-dimensional boundary is divided into uniform cubes to obtain a plurality of voxels. Then, the target voxel corresponding to each point cloud is determined based on the coordinate information of each point cloud in the completed point cloud data, and the corresponding point cloud filling value is filled into the target voxel. Further, the voxel with the point cloud is marked as 1, indicating that the voxel is occupied, and the voxel without the point cloud is marked as 0, indicating that the voxel is not occupied, to obtain the voxel occupancy features corresponding to the completed point cloud data.

[0123] Step 660, feature fusion is performed on the voxel occupancy features, the local geometric features and the neighborhood topological features to obtain the fusion features.

[0124] As an optional mode, after obtaining the occupancy features, the local geometric features and the neighborhood topological features corresponding to the completed point cloud data, the multi-head space-channel cross-attention mechanism can be used for feature fusion to obtain the fusion features with enhanced target detection feature expression. The fusion features are used as the input of the second detection network to further determine the target candidate box, so as to ensure the accuracy of the determined target candidate box.

[0125] Step 670, input the fusion feature of the completed point cloud data, the target encoding value, and the initial candidate box to a pre-trained second detection network, and output a target candidate box of the first object, wherein the first detection network and the second detection network contain different network layers, and the target candidate box is used to frame a three-dimensional region of the first object in the environment, and the three-dimensional region includes an orientation angle of the first object in the environment.

[0126] Step 680, determine a detection result of the first object according to the target candidate box.

[0127] The specific steps of steps 610-630 and steps 670-680 can be involved in steps 210-250, and will not be repeated here.

[0128] In the embodiment, by voxel processing and feature extraction on the completed point cloud data, voxel occupancy features, local geometric features, and neighborhood topological features are obtained, and then the voxel occupancy features, the local geometric features, and the neighborhood topological features are fused to improve the expression quality of the fusion features, ensure the accuracy of the determined target candidate box, and further improve the detection accuracy of the first object.

[0129] To realize the functions of the above-mentioned embodiments, the above-mentioned electronic device includes hardware structures and / or software modules corresponding to each function. Those skilled in the art should easily realize that the units and method steps of each example described in combination with the embodiments disclosed in the present application can be realized in the form of hardware or hardware and computer software. Whether a certain function is executed by hardware or computer software driven hardware depends on the specific application scenario and design constraints of the technical solution.

[0130] The above-mentioned embodiments describe in detail the target detection method provided by the embodiments of the present application. In other embodiments, the present application also provides a target detection device, Figure 9 is a block diagram of a target detection device according to an embodiment of the present application, as shown in Figure 9 The target detection device 700 includes an original point cloud data acquisition module 710, a point cloud completion module 720, an initial candidate box determination module 730, a target candidate box determination module 740, and a detection result determination module 750.

[0131] The original point cloud data acquisition module 710 is configured to acquire original point cloud data of an environment in which a first object is located. The point cloud completion module 710 is configured to perform point cloud completion on the original point cloud data to determine completed point cloud data. The initial candidate box determination module 730 is configured to input the completed point cloud data into a pre-trained first detection network to output an initial candidate box of the first object and a target encoding value. The target encoding value is used to indicate an orientation angle of the object in the environment, and the initial candidate box is used to frame a two-dimensional region of the first object in the environment. The target candidate box determination module 740 is configured to input fusion features of the completed point cloud data, the target encoding value, and the initial candidate box into a pre-trained second detection network to output a target candidate box of the first object. The first detection network and the second detection network include different network layers. The target candidate box is used to frame a three-dimensional region of the first object in the environment, and the three-dimensional region includes the orientation angle of the first object in the environment. The detection result determination module 750 is configured to determine a detection result of the first object according to the target candidate box.

[0132] In some embodiments, the initial candidate box determination module 730 includes an orientation angle determination sub-module configured to determine an orientation angle of the first object in the environment by performing orientation angle detection on the completed point cloud data by using the first detection network. The target encoding value determination sub-module is configured to determine the target encoding value by querying a mapping relationship according to the orientation angle. The mapping relationship is used to indicate an encoding value corresponding to each of a plurality of orientation angles.

[0133] In some embodiments, the target encoding value determination sub-module includes a first determination unit configured to determine a residual angle corresponding to the orientation angle and an angle width of the orientation angle according to the mapping relationship. A second determination unit is configured to determine the target encoding value according to the angle width, the residual angle, and the orientation angle.

[0134] In some embodiments, the point cloud completion module 720 includes a similarity determination sub-module configured to match the original point cloud data with each sample point cloud set in a sample point cloud library to determine a similarity between the original point cloud data and each sample point cloud set in the sample point cloud library. A point cloud completion sub-module is configured to determine a target sample point cloud according to the similarity, and perform point cloud completion on a region in the original point cloud data in which point cloud data is missing compared to the target sample point cloud based on the target sample point cloud to obtain the completed point cloud data.

[0135] In some embodiments, the similarity determination submodule comprises: a first similarity score determination unit configured to determine a first similarity score according to first coordinate information of the original point cloud data and second coordinate information of each sample point cloud set in the sample point cloud library; a second similarity score determination unit configured to determine a first bounding box corresponding to the original point cloud data and a second bounding box of each sample point cloud set in the sample point cloud library, and determine a second similarity score between the first bounding box and the second bounding box; an additional voxel number determination unit configured to determine an additional voxel number of each sample point cloud set in the sample point cloud library compared with the original point cloud data; and a matching score determination unit configured to determine the similarity according to the first similarity score, the second similarity score, and the additional voxel number.

[0136] In some embodiments, the initial candidate box determination module 730 comprises: a confidence determination submodule configured to determine a confidence corresponding to each point cloud in the completed point cloud data; a target detection point cloud data determination submodule configured to determine point cloud data corresponding to a confidence greater than a confidence threshold as target detection point cloud data; and an initial candidate box determination submodule configured to input the target detection point cloud data into a pre-trained first detection network to perform initial detection on a region where the first object is located, and output an initial candidate box of the first object.

[0137] In some embodiments, the target detection apparatus 700 further comprises: a feature extraction module configured to perform feature extraction on the completed point cloud data, and determine local geometric features and neighborhood topological features corresponding to the completed point cloud data; a voxel processing module configured to perform voxel processing on the completed point cloud data, and determine voxel occupancy features corresponding to the completed point cloud data; and a feature fusion module configured to perform feature fusion on the voxel occupancy features, the local geometric features, and the neighborhood topological features, and obtain the fusion features.

[0138] According to another aspect of embodiments of the present application, as Figure 10 As shown in FIG. 8, the electronic device 100 further comprises a processor 810 and one or more memories 820, the one or more memories 820 are configured to store program instructions executed by the processor 810, and the processor 810 executes the program instructions to implement the vehicle control method described above.

[0139] Further, the processor 810 can include one or more processing cores. The processor 810 runs or executes instructions, programs, code sets or instruction sets stored in the memory 820, and calls data stored in the memory 820. Alternatively, the processor 810 can be implemented in at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 810 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU is mainly used to process operating systems, user interfaces, and application programs; the GPU is used to be responsible for rendering and drawing display content; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor, but be realized by a separate communication chip.

[0140] According to an aspect of the present application, the present application also provides a computer readable storage medium, which can be contained in the cloud server described in the above embodiments, or can exist independently and not be assembled into the cloud server. The above computer readable storage medium carries computer readable instructions, which, when executed by a processor, implement the method in any of the above embodiments.

[0141] It should be noted that the computer readable medium shown in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination of the above.

[0142] The units described in the embodiments of the present application can be implemented in software, or in hardware, or in a combination of software and hardware. The units described can also be arranged in a processor. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0143] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the application following the general principles thereof and including such departures from the present disclosure as come within known use or custom in the art.

[0144] It is to be understood that the application is not limited to the precise construction already described above and shown in the drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application should only be limited by the claims appended hereto.

Claims

1. A target detection method, characterized in that, The method includes: Obtain the raw point cloud data of the environment where the first object is located; Perform point cloud completion on the original point cloud data to determine the completed point cloud data; The completed point cloud data is input into a pre-trained first detection network, which outputs an initial candidate box and a target encoding value for the first object. The target encoding is used to indicate the orientation angle of the object in the environment, and the initial candidate box is used to select a two-dimensional region of the first object in the environment. The fusion features of the completed point cloud data, the target encoding value, and the initial candidate box are input into a pre-trained second detection network to output the target candidate box of the first object. The first detection network and the second detection network contain different network layers. The target candidate box is used to select the three-dimensional region of the first object in the environment. The three-dimensional region includes the orientation angle of the first object in the environment. The detection result for the first object is determined based on the target candidate box.

2. The method according to claim 1, characterized in that, The step of inputting the completed point cloud data into a pre-trained first detection network and outputting the initial candidate bounding box and target encoding value of the first object includes: The orientation angle of the first object in the environment is determined by detecting the orientation angle of the completed point cloud data through the first detection network. The target encoded value is determined by querying the mapping relationship based on the orientation angle, wherein the mapping relationship is used to indicate the encoded value corresponding to each of the multiple orientation angles.

3. The method according to claim 2, characterized in that, The step of determining the target encoding value based on the orientation angle query mapping relationship includes: The residual angle corresponding to the orientation angle and the angular width of the orientation angle are determined according to the mapping relationship. The target encoding value is determined based on the angle width, the residual angle, and the orientation angle.

4. The method according to claim 1, characterized in that, The step of performing point cloud completion on the original point cloud data and determining the completed point cloud data includes: The original point cloud data is matched with each sample point cloud set in the sample point cloud library to determine the similarity between the original point cloud data and each sample point cloud set in the sample point cloud library. The target sample point cloud is determined based on the similarity, and the regions in the original point cloud data that lack point cloud data compared to the target sample point cloud are filled in based on the target sample point cloud to obtain the filled point cloud data.

5. The method according to claim 4, characterized in that, The step of matching the original point cloud data with each sample point cloud set in the sample point cloud library to determine the similarity between the original point cloud data and each sample point cloud set in the sample point cloud library includes: A first similarity score is determined based on the first coordinate information of the original point cloud data and the second coordinate information of each sample point cloud set in the sample point cloud library. Determine the first bounding box corresponding to the original point cloud data and the second bounding box of each sample point cloud set in the sample point cloud library, and determine the second similarity score between the first bounding box and the second bounding box; Determine the number of additional voxels for each sample point cloud set in the sample point cloud library compared to the original point cloud data; The similarity is determined based on the first similarity score, the second similarity score, and the number of additional voxels.

6. The method according to any one of claims 1-5, characterized in that, The step of inputting the completed point cloud data into a pre-trained first detection network and outputting the initial candidate bounding box and target encoding value of the first object includes: Determine the confidence level for each point cloud in the completed point cloud data; The point cloud data with a confidence level greater than the confidence threshold are identified as target detection point cloud data; The target detection point cloud data is input into a pre-trained first detection network to perform initial detection on the region where the first object is located, and the initial candidate box of the first object is output.

7. The method according to any one of claims 1-5, characterized in that, Before inputting the fused features of the completed point cloud data, the target encoding value, and the initial candidate box into the pre-trained second detection network to output the target candidate box of the first object, the method further includes: Feature extraction is performed on the completed point cloud data to determine the local geometric features and neighborhood topological features corresponding to the completed point cloud data; The completed point cloud data is processed by voxel processing to determine the voxel occupancy features corresponding to the completed point cloud data; The voxel occupancy feature, the local geometric feature, and the neighborhood topological feature are fused to obtain the fused feature.

8. A target detection device, characterized in that, The device includes: The raw point cloud data acquisition module is used to acquire the raw point cloud data of the environment where the first object is located; The point cloud completion module is used to complete the original point cloud data and determine the completed point cloud data; The initial candidate box determination module is used to input the completed point cloud data into a pre-trained first detection network and output the initial candidate box and target encoding value of the first object, wherein the target encoding is used to indicate the orientation angle of the object in the environment, and the initial candidate box is used to select the two-dimensional region of the first object in the environment; The target candidate box determination module is used to input the fusion features of the completed point cloud data, the target encoding value, and the initial candidate box into a pre-trained second detection network, and output the target candidate box of the first object. The first detection network and the second detection network contain different network layers. The target candidate box is used to select the three-dimensional region of the first object in the environment. The three-dimensional region includes the orientation angle of the first object in the environment. The detection result determination module is used to determine the detection result of the first object based on the target candidate box.

9. An electronic device, characterized in that, The electronic device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images