Model training method, target detection method, device, equipment, vehicle and medium

Abstract

The present application relates to a model training method, a target detection method, an apparatus, a device, a vehicle, and a medium. The model training method includes: obtaining a first point cloud image containing a first target object collected by a vehicle; the first target object has label information, the label information includes a corner point label and a heading angle label of the first target object, and the corner point label is generated based on the distance between the corner point and the center point; using a target detection model to be trained to detect the first target object, and obtain detection information of the first target object, the detection information includes a corner point prediction value and a heading angle prediction value; determining a target loss value based on the label information and the detection information; based on the target loss value, performing at least one parameter update on the target detection model to be trained to obtain a trained target detection model. The embodiments of the present application improve the efficiency of model training and the accuracy of model detection.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and specifically to a model training method, target detection method, device, equipment, vehicle and medium. Background Art

[0002] With the rapid development of technologies such as autonomous driving, intelligent robots, and drones, three-dimensional object detection, as one of the key technologies for environmental perception, has become a research hotspot in the field of computer vision. The goal of 3D object detection is to accurately identify and locate target objects from two-dimensional images or three-dimensional point cloud data, and to infer the spatial position, size, and posture (i.e., direction and angle) of the object. In related technologies, target detection relies on lidar or depth cameras to obtain three-dimensional information, which may make depth estimation susceptible to environmental changes. How to effectively fuse different sensor data and directly process the point cloud data collected by the sensor involves a large amount of calculation, resulting in high resource overhead and other issues. Summary of the Invention

[0003] Embodiments of the present application provide a model training method, target detection method, device, equipment, vehicle and medium.

[0004] The technical solutions adopted in the embodiments of this application are as follows:

[0005] The present invention provides a model training method, comprising:

[0006] Acquire a first point cloud image containing a first target object; the first point cloud image is acquired by vehicle acquisition, the first target object having label information, the label information including corner point labels of the first target object and a heading angle label of the first target object, the corner point labels of the first target object being generated based on distances between corner points of the first target object and a center point of the first target object;

[0007] Detecting a first target object in the first point cloud image using the target detection model to be trained to obtain detection information of the first target object, where the detection information of the first target object includes a predicted value of a corner point of the first target object and a predicted value of a heading angle of the first target object;

[0008] determining a target loss value based on the label information of the first target object and the detection information of the first target object;

[0009] Based on the target loss value, the parameters of the target detection model to be trained are updated at least once to obtain a trained target detection model.

[0010] According to the above technical means, first, a first point cloud image containing a first target object is obtained; the first point cloud image is obtained by vehicle acquisition, and the first target object has label information, including corner point labels and heading angle labels of the first target object; the corner point labels are generated based on the distance between the corner points of the first target object and the center point of the first target object; and the heading angle labels are generated based on the corner points of the first target object. Secondly, a target detection model to be trained is used to detect the first target object in the first point cloud image to obtain detection information of the first target object, including predicted corner point values ​​and predicted heading angle values ​​of the first target object, and a target loss value is determined based on the label information and the detection information. Finally, based on the target loss value, the parameters of the target detection model to be trained are updated at least once to obtain a trained target detection model. In this way, during the model training process, the target detection model to be trained is supervisedly trained based on the corner point labels and heading angle labels of the first target object, so that the trained target detection model can accurately predict the corner points and heading angle of the first target object, and the corner point labels are generated based on the distance between the corner points and the center point of the first target object, thereby improving the accuracy of the model's corner point prediction.

[0011] Furthermore, the label information also includes a size label of the first target object and a center point label of the first target object, and the detection information of the first target object also includes a size prediction value of the first target object and a center point prediction value of the first target object; the above-mentioned determination of the target loss value based on the label information of the first target object and the detection information of the first target object includes: determining the size regression loss based on the size prediction value of the first target object and the size label of the first target object; determining the corner point regression loss based on the corner point prediction value of the first target object and the corner point label of the first target object; determining the center point regression loss based on the center point prediction value of the first target object and the center point label of the first target object; determining the heading angle regression loss based on the heading angle prediction value of the first target object and the heading angle label of the first target object; determining the target loss value based on the size regression loss, the corner point regression loss, the center point regression loss and the heading angle regression loss.

[0012] According to the above technical means, the size regression loss is determined based on the size prediction value and size label of the first target object; the corner regression loss is determined based on the corner prediction value and corner label of the first target object; the center point regression loss is determined based on the center point prediction value and center point label of the first target object; the heading angle regression loss is determined based on the heading angle prediction value and heading angle label of the first target object; and the target loss value is determined based on the size regression loss, corner regression loss, center point regression loss, and heading angle regression loss. In this way, during the model training process, the size regression loss, corner regression loss, center point regression loss, and heading angle regression loss of the first target object are comprehensively considered, so that the model can accurately predict the size, corners, center point, and heading angle of the first target object.

[0013] Furthermore, the above-mentioned determination of the target loss value based on the size regression loss, corner regression loss, center point regression loss and heading angle regression loss includes: respectively determining the first weight coefficient of the size regression loss, the second weight coefficient of the corner regression loss, the third weight coefficient of the center point regression loss and the fourth weight coefficient of the heading angle regression loss; determining the target loss value based on the first weight coefficient, the second weight coefficient, the third weight coefficient, the fourth weight coefficient, the size regression loss, the corner regression loss, the center point regression loss and the heading angle regression loss.

[0014] According to the above technical means, the first weight coefficient of the size regression loss, the second weight coefficient of the corner regression loss, the third weight coefficient of the center point regression loss, and the fourth weight coefficient of the heading angle regression loss are respectively determined; and the target loss value is determined based on the first weight coefficient, the second weight coefficient, the third weight coefficient, the fourth weight coefficient, the size regression loss, the corner regression loss, the center point regression loss, and the heading angle regression loss. In this way, the target loss is obtained based on the weight coefficients corresponding to the size regression loss, the corner regression loss, the center point regression loss, and the heading angle regression loss, respectively. The target loss can more accurately measure the error between the predicted value and the true value of the target detection model, thereby more accurately adjusting the parameters of the target detection model to be trained, and obtaining a more accurate trained target detection model.

[0015] Furthermore, the above method also includes: determining the corner points of the first target object based on the size information of the first target object, the center point of the first target object and the heading angle of the first target object; and determining the corner point labels of the first target object based on the corner points of the first target object and the center point of the first target object.

[0016] According to the above technical approach, the corner points of the first target object are determined based on the size information of the first target object, the center point of the rectangular frame, and the heading angle. Furthermore, the corner point labels of the first target object are determined based on the corner points and the center point of the first target object. In this way, the distance between the corner points and the center point of the first target object is used as the corner point label to train the target detection model to be trained. This makes the trained target detection model more accurate in predicting corner points, thereby improving the accuracy of the trained target detection model.

[0017] Furthermore, the above-mentioned determining the corner points of the first target object based on the size information of the first target object, the center point of the first target object and the heading angle of the first target object includes: determining the corner points of the first target object in the local coordinate system based on the size information of the first target object and the center point of the first target object; the local coordinate system is a coordinate system with the center point of the first target object as the coordinate origin; determining the rotation matrix based on the heading angle; and rotating and translating the corner points of the first target object in the local coordinate system based on the rotation matrix to obtain the corner points of the first target object in the world coordinate system.

[0018] According to the above technical approach, the corner points of the first target object are determined in a local coordinate system based on the size information and center point of the first target object. The local coordinate system is a coordinate system with the center point of the first target object as the coordinate origin. A rotation matrix is ​​determined based on the heading angle. The corner points of the first target object in the local coordinate system are rotated and translated based on the rotation matrix to obtain the corner points of the first target object in the world coordinate system. In this way, the coordinates of the corner points of the first target object in the world coordinate system can be more accurately determined based on the size information, center point coordinates, and heading angle.

[0019] Furthermore, the corner points of the first target object include a first corner point and a second corner point, and the above method also includes: determining a first difference between the horizontal coordinate of the first corner point and the horizontal coordinate of the second corner point; determining a second difference between the vertical coordinate of the first corner point and the vertical coordinate of the second corner point; and generating a heading angle label of the first target object based on the first difference and the second difference.

[0020] According to the above technical approach, a heading angle label is generated based on a first difference between the abscissa of the first corner point and the abscissa of the second corner point, and a second difference between the ordinate of the first corner point and the ordinate of the second corner point. In this way, a heading angle label can be generated based on the distance between the first corner point and the second corner point represented by the first difference and the second difference, simplifying the heading angle label generation process and improving model training efficiency.

[0021] Furthermore, generating the heading angle label of the first target object based on the first difference and the second difference includes: determining a ratio between the first difference and the second difference; and generating the heading angle label of the first target object based on an arctangent value of the ratio.

[0022] According to the above technical approach, a heading angle tag for the first target object is generated based on the inverse tangent of the ratio between the first difference and the second difference. This simplifies the process of generating the heading angle tag by using the inverse tangent function and the distance between the corner points of the first target object.

[0023] The present invention provides a method for detecting a target, including:

[0024] Acquire a second point cloud image to be detected;

[0025] The target detection model is used to detect the second target object in the second point cloud image to obtain detection information of the second target object; wherein the target detection model is obtained by using the above-mentioned model training method.

[0026] Using the aforementioned technical approach, a second point cloud image to be detected is obtained. Using the object detection model obtained using the aforementioned model training method, the second target object in the second point cloud image is detected to obtain detection information of the second target object. Thus, because the trained object detection model is obtained through supervised training based on the corner point labels and heading angle labels, the trained object detection model can accurately detect the corner points and heading angles of the second target object.

[0027] The present invention provides a model training device, comprising:

[0028] A first acquisition module is configured to acquire a first point cloud image containing a first target object; the first point cloud image is acquired by vehicle acquisition, and the first target object has label information, the label information including a corner point label of the first target object and a heading angle label of the first target object; the corner point label of the first target object is generated based on a distance between a corner point of the first target object and a center point of the first target object;

[0029] a first detection module, configured to detect a first target object in the first point cloud image using the target detection model to be trained, and obtain detection information of the first target object, the detection information of the first target object including a predicted value of a corner point of the first target object and a predicted value of a heading angle of the first target object;

[0030] a first determining module, configured to determine a target loss value based on label information of the first target object and detection information of the first target object;

[0031] The parameter updating module is used to update the parameters of the target detection model to be trained at least once based on the target loss value to obtain the trained target detection model.

[0032] The present invention provides a target detection device, including:

[0033] A second acquisition module is used to acquire a second point cloud image to be detected;

[0034] The second detection module is used to detect the second target object in the second point cloud image using the target detection model to obtain detection information of the second target object; wherein the target detection model is obtained using the above-mentioned model training method.

[0035] An embodiment of the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0036] An embodiment of the present application provides a vehicle, comprising the above-mentioned computer device.

[0037] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements some or all of the steps in the above method when executed by a processor.

[0038] Beneficial effects of this application:

[0039] (1) During the model training process, the target detection model to be trained is supervised by using the corner point labels and heading angle labels of the first target object, so that the trained target detection model can accurately predict the corner points and heading angles of the first target object. In addition, the corner point labels are generated by using the distance between the corner points and the center point of the first target object, so that the trained target detection model can more accurately predict the corner points, thereby improving the accuracy of the model's corner point prediction;

[0040] (2) During the model training process, the size regression loss, corner regression loss, center point regression loss, and heading angle regression loss of the first target object are comprehensively considered, so that the model can accurately predict the size, corner points, center point, and heading angle of the first target object;

[0041] (3) The target loss is obtained according to the weight coefficients corresponding to the size regression loss, corner regression loss, center point regression loss and heading angle regression loss, so that the target loss can more accurately measure the error between the predicted value and the true value of the target detection model, thereby more accurately adjusting the parameters of the target detection model to be trained, and obtaining a more accurate trained target detection model;

[0042] (4) The heading angle label is generated by the distance between the first corner point and the second corner point represented by the first difference and the second difference, which simplifies the generation process of the heading angle label and improves the efficiency of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the implementation flow of a model training method proposed in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of an implementation flow of a target detection method proposed in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of a corner point label design proposed in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of a heading angle tag design proposed in an embodiment of the present application;

[0047] Figure 5 A schematic diagram of the structure of a model training device proposed in an embodiment of the present application;

[0048] Figure 6 A schematic diagram of the structure of a target detection device proposed in an embodiment of the present application;

[0049] Figure 7 A hardware entity diagram of a computer device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will describe the embodiments of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand the other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for the purpose of illustrating the present application and are not intended to limit the scope of protection of the present application.

[0051] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0052] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0053] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0055] The present application provides a model training method that can be executed by a processor of a computer device. During implementation, the computer device may include, but is not limited to, at least one of a vehicle computer, a robot, a laptop computer, a tablet computer, a desktop computer, a large-screen device, a server, and a mobile device. Figure 1 A schematic diagram of the implementation flow of a model training method proposed in an embodiment of the present application is shown in FIG. Figure 1 As shown, the model training method includes the following steps S101 to S104, wherein:

[0056] Step S101: Acquire a first point cloud image containing a first target object; the first point cloud image is acquired by vehicle acquisition, and the first target object has label information, the label information including corner point labels of the first target object and a heading angle label of the first target object; the corner point labels of the first target object are generated based on the distance between the corner points of the first target object and the center point of the first target object;

[0057] Here, the first point cloud image includes a first target object, which may be a target object of different types or with different characteristics. For example, the first target object may be a target object in a different scene, or the geometric shape, spatial distribution, etc. of the first target object may be different.

[0058] In some embodiments, the first point cloud image may be an image collected by fusing multiple sensors such as an RGB camera, a LiDAR, a depth camera, and a radar installed on the vehicle.

[0059] In some embodiments, datasets such as KITTI, nuScenes, and Waymo collected by a vehicle may be preprocessed by performing data format alignment, size adjustment, image enhancement, label information generation, and the like to obtain a first point cloud image.

[0060] In some embodiments, the first target object has label information, and the label information may include a corner point label of the first target object and a heading angle label of the first target object, wherein:

[0061] The corner labels of the first target object are used to supervise the object detection model to be trained in predicting the coordinates of the first target object's corners. A corner of the first target object is the intersection of two or more edges of the first target object, or a point where the curvature of the target object is locally maximum. Corner points are typically located at the corners of the edges of the first target object and have unique geometric characteristics.

[0062] The heading angle label of the first target object is used to supervise the object detection model being trained to predict the heading angle of the first target object. The heading angle of the first target object describes the orientation angle of a three-dimensional object in the two-dimensional image plane, such as the direction of a vehicle, pedestrian, or obstacle. The heading angle of the first target object is the angle between the first target object's forward direction and the image coordinate system (e.g., the camera coordinate system).

[0063] In some embodiments, the corner point labels are generated based on the distance between the corner points and the center point of the first target object. In other embodiments, the corner point labels may be generated based on the coordinates of the corner points of the first target object.

[0064] In some embodiments, the heading angle tag is generated based on the relative positions of the corner points of the first target object. In other words, the heading angle tag can be generated based on the geometric relationship between the four corner points of the first target object. This simplifies the calculation process of the heading angle tag, making subsequent heading angle estimation more accurate.

[0065] In some embodiments, the heading angle tag may be generated by geometrically calculating three-dimensional data collected by sensors on the vehicle.

[0066] Step S102: Detecting a first target object in the first point cloud image using the target detection model to be trained to obtain detection information of the first target object, where the detection information includes a size prediction value of the first target object, corner prediction values ​​of the first target object, a center prediction value of the first target object, and a heading angle prediction value of the first target object.

[0067] Here, the target detection model to be trained can be any model that can perform the target detection task, for example, it can be a VoxelNet model, a SECOND model, a LAM3D model, etc.

[0068] In some embodiments, the target detection model to be trained is used to perform feature extraction on the first target object in the first point cloud image to obtain detection information such as corner points and heading angles of the first target object. The detection information includes predicted corner point values ​​and heading angle prediction values ​​of the first target object.

[0069] In some embodiments, independent network branches in the target detection model to be trained are used to detect the corner points and heading angles of the first target object, respectively. For example, a corner point detection network is used to detect the corner points of the first target object, and a heading angle detection network is used to detect the heading angle of the first target object.

[0070] Step S103: determining a target loss value based on the label information of the first target object and the detection information of the first target object;

[0071] In some embodiments, based on the label information and the detection information, the corner point regression loss and the heading angle regression loss corresponding to the corner point and heading angle of the first target object are respectively determined, and then the target loss is determined based on the corner point regression loss and the heading angle regression loss.

[0072] In some implementations, a corner point regression loss is determined based on the corner point labels and corner point prediction values ​​of the first target object; and a heading angle regression loss is determined based on the heading angle label and heading angle prediction value of the first target object.

[0073] Step S104: Based on the target loss value, update the parameters of the target detection model to be trained at least once to obtain a trained target detection model.

[0074] In some embodiments, parameters of the target detection model to be trained are updated to minimize the target loss value, thereby making the target detection result using the target detection model to be trained more accurate.

[0075] In some embodiments, parameters of the target detection model to be trained are updated according to the target loss to minimize the target loss, and the trained target detection model is obtained by updating the parameters of the target detection model to be trained multiple times.

[0076] In some embodiments, when a preset number of training times is reached during the training of the target detection model, the training of the target detection model is stopped to obtain a trained target detection model.

[0077] In some embodiments, during the training of the target detection model, when the target loss satisfies a preset convergence condition, the training of the target detection model is stopped to obtain a trained target detection model.

[0078] In an embodiment of the present application, first, a first point cloud image containing a first target object is acquired; the first point cloud image is acquired by vehicle acquisition, and the first target object has label information, the label information including corner point labels and heading angle labels of the first target object; the corner point labels are generated based on the distance between the corner points of the first target object and the center point of the first target object; the heading angle labels are generated based on the corner points of the first target object; secondly, a target detection model to be trained is used to detect the first target object in the first point cloud image to obtain detection information of the first target object, the detection information of the first target object including corner point prediction values ​​and heading angle prediction values ​​of the first target object, and a target loss value is determined based on the label information and the detection information; finally, based on the target loss value, the parameters of the target detection model to be trained are updated at least once to obtain a trained target detection model. In this way, during the model training process, the target detection model to be trained is supervisedly trained based on the corner point labels and heading angle labels of the first target object, so that the trained target detection model can accurately predict the corner points and heading angle of the first target object, and the corner point labels are generated based on the distance between the corner points and the center point of the first target object, thereby improving the accuracy of the model's corner point prediction.

[0079] In some embodiments, the label information further includes a size label of the first target object and a center point label of the first target object, and the detection information of the first target object further includes a size prediction value of the first target object and a center point prediction value of the first target object; the above-mentioned step S103 may include the following steps S1031 to S1035:

[0080] Step S1031: determining a size regression loss based on the size prediction value of the first target object and the size label of the first target object;

[0081] In some embodiments, the label information may further include a size label for the first target object, where the size label for the first target object is used to supervise the trained object detection model in predicting the size of the rectangular bounding box of the first target object. The size of the first target object may include the length, width, and height of the rectangular bounding box of the first target object.

[0082] The size label is generated based on the size (length, width, and height) of the rectangular frame of the first target object. In one example, the size information of the first target object can be manually recorded based on the rectangular frame of the first target object to generate a size label. In another example, a pre-trained model is used to generate a preliminary rectangular frame and an initial size label by combining an automatic detection algorithm and manual correction, and then manual correction and confirmation are performed to generate a final size label.

[0083] The target detection model to be trained is used to detect the first target object in the first point cloud image to obtain a size prediction value, and a size regression loss is determined based on the size prediction value and the size label.

[0084] In some embodiments, to enable the trained object detection model to accurately detect the size of the first target object, a mean squared error loss function (L2 loss) may be used to calculate the target object's size regression loss. The L2 loss can effectively reduce the gap between the predicted size (size prediction value) and the true size (size label), enabling the trained object detection model to accurately detect the size of the first target object, thereby determining the first target object's three-dimensional size.

[0085] Dimensional regression loss L size The calculation formula for is shown in formula (1):

[0086] (1);

[0087] In formula (1), is the size prediction value, is the size label (the actual value of the size).

[0088] In some embodiments, the size prediction value obtained by the object detection model to be trained and the pre-generated size label are substituted into formula (1) to obtain the size regression loss.

[0089] In some implementations, a size regression loss is determined based on the size prediction value of the first target object and an offset between the size label.

[0090] Step S1032: determining a corner regression loss based on the corner prediction value of the first target object and the corner label of the first target object;

[0091] In some embodiments, to enable the trained object detection model to accurately detect the corners of the first target object, a smooth L1 loss is used to calculate the corner regression loss. The smooth L1 loss behaves as an L2 loss for small errors and as a mean absolute error function (L1 loss) for large errors. Therefore, using the smooth L1 loss to calculate the corner regression loss can avoid the excessive penalty imposed by the L2 loss for large errors while accelerating convergence for small errors.

[0092] Corner regression loss The calculation of is shown in formula (2):

[0093] (2);

[0094] In formula (2), Corner point prediction values, i.e., the corner point coordinates predicted by the target detection model to be trained; The actual corner coordinates are determined based on the corner labels.

[0095] In some embodiments, the corner point prediction values ​​obtained by the target detection model to be trained and the pre-generated corner point labels are substituted into formula (2) to obtain the corner point regression loss.

[0096] In some embodiments, the corner point label is the distance between the corner point and the center point, and the true corner point coordinates can be determined based on the corner point label and the center point coordinates. In some embodiments, the corner point regression loss is determined based on the corner point prediction value and the true corner point coordinates.

[0097] In some embodiments, the distance between the corner point and the center point is used as the regression target. In this case, the corner point prediction value is the distance prediction value between the corner point and the center point. The corner point regression loss is determined based on the distance prediction value and the corner point label.

[0098] In some implementations, a corner regression loss is determined based on the predicted values ​​of the corners of the first target object and the offsets of the corner labels.

[0099] Step S1033: determining a center point regression loss based on the center point prediction value of the first target object and the center point label of the first target object;

[0100] In some embodiments, the label information may also include a center point label for the first target object. The center point label for the first target object is used to supervise the prediction of the coordinates of the center point of the first target object by the target detection model to be trained. The center point of the first target object refers to the geometric center position of the first target object on a two-dimensional plane. It is the midpoint of the first target object in the horizontal and vertical directions and represents the center of symmetry of the first target object.

[0101] The center point label is generated according to the center point coordinates of the first target object.

[0102] The target detection model to be trained is used to detect the first target object in the first point cloud image to obtain a size prediction value.

[0103] In some embodiments, the center point regression loss is determined based on the center point prediction value and the center point label, and the parameters of the target detection model to be trained are updated based on the center point regression loss, so that the target detection model to be trained can accurately detect the center point of the first target object.

[0104] Center point regression loss The calculation of is shown in formula (3):

[0105] (3);

[0106] In formula (3), is the center point prediction value, is the center point label (real center point coordinates).

[0107] In some embodiments, the center point prediction value obtained by the target detection model to be trained and the pre-generated center point label are substituted into formula (3) to obtain the center point regression loss.

[0108] In some implementations, a center point regression loss is determined based on an offset between a center point prediction value of the first target object and a center point label.

[0109] Step S1034: determining a heading angle regression loss based on the heading angle prediction value of the first target object and the heading angle label of the first target object;

[0110] In some embodiments, a heading angle regression loss is determined based on the heading angle prediction value and the heading angle label, so that the parameters of the target detection model to be trained are updated based on the heading angle regression loss, so that the target detection model to be trained can accurately detect the heading angle of the first target object.

[0111] The heading angle is crucial information about the first target's posture. Especially in dynamic detection, the accuracy of the heading angle estimation directly impacts the spatial positioning of the first target and subsequent tasks (such as path planning). To ensure accurate heading angle detection, a smooth L1 loss is used to calculate the heading angle regression loss.

[0112] Heading angle The calculation of regression loss is shown in formula (4):

[0113] (4);

[0114] In formula (4), is the predicted heading angle, It is the heading angle label (real heading angle).

[0115] In some embodiments, the heading angle prediction value obtained by the target detection model to be trained and the pre-generated heading angle label are substituted into formula (4) to obtain the heading angle regression loss.

[0116] In some implementations, a heading angle regression loss is determined based on the heading angle prediction value of the first target object and an offset of the heading angle label.

[0117] Step S1035: Determine the target loss value based on the size regression loss, the corner point regression loss, the center point regression loss, and the heading angle regression loss.

[0118] In some embodiments, the target loss is determined based on the size regression loss, corner point regression loss, center point regression loss and heading angle regression loss, as well as the weight coefficients corresponding to the size regression loss, corner point regression loss, center point regression loss and heading angle regression loss, respectively.

[0119] In some embodiments, the size regression loss, corner regression loss, center point regression loss, and heading angle regression loss are summed to determine the target loss.

[0120] In some implementations, the weight coefficients corresponding to the size regression loss, corner regression loss, center point regression loss, and heading angle regression loss can be adaptively adjusted. For example, if the error between the size prediction value and the size label is large, the weight coefficient corresponding to the size regression loss can be increased to improve the accuracy of the target object size prediction.

[0121] In an embodiment of the present application, a size regression loss is determined based on the size prediction value and size label of the first target object; a corner regression loss is determined based on the corner prediction value and corner label of the first target object; a center regression loss is determined based on the center prediction value and center label of the first target object; a heading regression loss is determined based on the heading angle prediction value and heading angle label of the first target object; and a target loss value is determined based on the size regression loss, corner regression loss, center regression loss, and heading regression loss. In this way, during the model training process, the size regression loss, corner regression loss, center regression loss, and heading regression loss of the first target object are comprehensively considered, so that the model can accurately predict the size, corners, center points, and heading angle of the first target object.

[0122] In some embodiments, the above step S1035 may include the following steps S10351 to S10352:

[0123] Step S10351: respectively determining a first weight coefficient of the size regression loss, a second weight coefficient of the corner point regression loss, a third weight coefficient of the center point regression loss, and a fourth weight coefficient of the heading angle regression loss;

[0124] In some implementations, the first weight coefficient, the second weight coefficient, the third weight coefficient, and the fourth weight coefficient may be set based on empirical values.

[0125] In some embodiments, the first weight coefficient, the second weight coefficient, the third weight coefficient, and the fourth weight coefficient can be adjusted according to the size regression loss, corner regression loss, center point regression loss, and heading angle regression loss of the first target object to optimize the proportion of the size regression loss, corner regression loss, center point regression loss, and heading angle regression loss.

[0126] Step S10352: Determine the target loss value based on the first weight coefficient, the second weight coefficient, the third weight coefficient, the fourth weight coefficient, the size regression loss, the corner point regression loss, the center point regression loss and the heading angle regression loss.

[0127] In some embodiments, target loss The calculation formula can be found in formula (5):

[0128] (5);

[0129] In formula (5), is the first weight coefficient; is the second weight coefficient; is the third weight coefficient; is the fourth weight coefficient; by adjusting these weight coefficients, the proportion of corner regression loss, size regression loss, center point regression loss and heading angle regression loss can be balanced to ensure that the accuracy of each regression task is appropriately improved.

[0130] In an embodiment of the present application, a first weight coefficient of the size regression loss, a second weight coefficient of the corner regression loss, a third weight coefficient of the center point regression loss, and a fourth weight coefficient of the heading angle regression loss are determined respectively; and a target loss value is determined based on the first weight coefficient, the second weight coefficient, the third weight coefficient, the fourth weight coefficient, the size regression loss, the corner regression loss, the center point regression loss, and the heading angle regression loss. In this way, the target loss is obtained based on the weight coefficients corresponding to the size regression loss, the corner regression loss, the center point regression loss, and the heading angle regression loss, respectively, so that the target loss can more accurately measure the error between the predicted value and the true value of the target detection model, thereby more accurately adjusting the parameters of the target detection model to be trained, and obtaining a more accurate trained target detection model.

[0131] In some embodiments, the above method further includes the following steps S105 to S106:

[0132] Step S105: determining a corner point of the first target object based on the size information of the first target object, the center point of the rectangular frame of the first target object, and the heading angle of the first target object;

[0133] In some implementations, the corner point is a function of the center point, size information, and heading angle of the first target object. The coordinates of the corner point of the first target object can be calculated through geometric operations.

[0134] In some implementations, the corner points of the first target object may be directly predicted based on the rectangular frame of the first target object, thereby obtaining the coordinates of the corner points of the first target object.

[0135] In some implementations, the corner points of the first target object may be represented by corner point coordinates.

[0136] In some implementations, the coordinates of the corner points of the first target object are determined using trigonometric functions according to the size information, the center point, and the heading angle of the first target object.

[0137] In some embodiments, the coordinates of the corner points of the first target object in the local coordinate system can be first determined. A rotation matrix can then be determined based on the heading angle. Rotation and translation can then be performed based on the rotation matrix and the center point coordinates to obtain the coordinates of the corner points of the first target object in the world coordinate system. The local coordinate system is a coordinate system with the center point of the first target object as its origin.

[0138] In some embodiments, the corner coordinates of the first target object in the camera coordinate system may be determined first, and then the first target object may be rotated according to a rotation matrix between the camera coordinate system and the world coordinate system to obtain the corner coordinates of the first target object in the world coordinate system.

[0139] In some implementations, the corner coordinates of the first target object in the world coordinate system are mapped to a two-dimensional plane to obtain the corner coordinates of the first target object.

[0140] Step S106: determining corner point labels of the first target object based on the corner points of the first target object and the center point of the first target object.

[0141] In some embodiments, during the training process of the target detection model, corner point labels are generated based on the distance between the corner points and the center point. By regressing the distance between the corner points and the center point, not only can the target detection model's prediction of the corner points be improved, but also the prediction of the center point can be improved.

[0142] In this embodiment of the present application, the corner points of the first target object are determined based on the size information of the first target object, the center point of the rectangular frame, and the heading angle. Furthermore, the corner point labels of the first target object are determined based on the corner points and the center point of the first target object. In this way, the distance between the corner points and the center point of the first target object is used as the corner point label to train the target detection model to be trained. This makes the trained target detection model more accurate in predicting corner points, thereby improving the accuracy of the trained target detection model.

[0143] In some embodiments, the above step S105 may include the following steps S1051 to S1053:

[0144] Step S1051: determining corner points of the first target object in a local coordinate system based on the size information of the first target object and the center point of the first target object; the local coordinate system is a coordinate system with the center point of the first target object as a coordinate origin;

[0145] Here, the local coordinate system is a coordinate system with the center point as the coordinate origin.

[0146] In some embodiments, the coordinates of the corner points of the target object in the local coordinate system can be determined based on the size information. That is, based on the length, width, and height in the size information, the coordinates of the corner points of the first target object in the local coordinate system are determined with the center point as the origin of the coordinate system.

[0147] In some embodiments, the corner point can be regarded as the offset of the center point in the local coordinate system. Therefore, the corner point offset of the first target object in the local coordinate system, that is, the corner point coordinates of the first target object in the local coordinate system, can be determined based on the center point coordinates and the length, width and height of the first target object.

[0148] Step S1052: Determine a rotation matrix based on the heading angle;

[0149] In some embodiments, the heading angle of the first target object can be defined as the rotation of the local coordinate system relative to the world coordinate system. Therefore, according to the rotation matrix corresponding to the heading angle and the corner point offset, the corner point coordinates are translated and rotated to obtain the corner point coordinates of the first target object in the world coordinate system.

[0150] Step S1053: Based on the rotation matrix, rotate and translate the corner points of the first target object in the local coordinate system to obtain the corner points of the first target object in the world coordinate system.

[0151] In some implementations, the coordinates of the corner point in the local coordinate system are rotated and translated according to a rotation matrix corresponding to the heading angle to obtain the coordinates of the corner point in the world coordinate system.

[0152] In the embodiment of the present application, the corner points of the first target object are determined in a local coordinate system based on the size information and center point of the first target object; the local coordinate system is a coordinate system with the center point of the first target object as the coordinate origin; a rotation matrix is ​​determined based on the heading angle; and the corner points of the first target object in the local coordinate system are rotated and translated based on the rotation matrix to obtain the corner points of the first target object in the world coordinate system. In this way, the coordinates of the corner points of the first target object in the world coordinate system can be more accurately determined based on the size information, center point coordinates, and heading angle.

[0153] In some embodiments, the corner points of the first target object include a first corner point and a second corner point; the above method further includes steps S107 to S109:

[0154] Step S107: determining a first difference between the abscissa of the first corner point and the abscissa of the second corner point;

[0155] Here, the first corner point and the second corner point are two corner points among all corner points of the first target object.

[0156] In some embodiments, the first corner point and the second corner point may be two adjacent corner points of the first target object. In some embodiments, the first corner point and the second corner point may be two non-adjacent corner points of the first target object.

[0157] In some embodiments, the first difference may represent a distance between the first corner point and the second corner point in the horizontal coordinate direction.

[0158] Step S108: determining a second difference between the longitudinal coordinate of the first corner point and the longitudinal coordinate of the second corner point;

[0159] In some embodiments, the second difference may represent the distance between the first corner point and the second corner point in the vertical coordinate direction.

[0160] Step S109: generating a heading angle label of the first target object based on the first difference and the second difference.

[0161] In some implementations, a heading angle tag may be generated using the distance between the first corner point and the second corner point represented by the first difference and the second difference.

[0162] In some implementations, a heading angle tag may be generated based on the corner point coordinates of the adjacent first corner point and the corner point coordinates of the adjacent second corner point.

[0163] In some implementations, a heading angle tag may be generated based on the corner point coordinates of a first corner point and a second corner point that are not adjacent to each other.

[0164] In some implementations, a heading angle tag may be generated based on the coordinates of four corner points of the first target object.

[0165] In the embodiment of the present application, a heading angle label is generated based on a first difference between the abscissa of a first corner point and the abscissa of a second corner point, and a second difference between the ordinate of the first corner point and the ordinate of the second corner point. In this way, a heading angle label can be generated based on the distance between the first corner point and the second corner point represented by the first difference and the second difference, simplifying the heading angle label generation process and thereby improving model training efficiency.

[0166] In some embodiments, the above step S109 may include the following steps S1091 to S1092:

[0167] Step S1091: determining a ratio between the first difference and the second difference;

[0168] Step S1092: Generate a heading angle label of the first target object based on the arctangent value of the ratio.

[0169] In some embodiments, the heading angle tag is obtained by using the inverse tangent function according to the ratio of the second difference to the first difference. The calculation of is shown in formula (6):

[0170] (6);

[0171] In formula (6), is the horizontal coordinate of the first corner point; is the horizontal coordinate of the second corner point; is the ordinate of the first corner point; is the ordinate of the second corner point.

[0172] In this embodiment of the present application, a heading angle tag for the first target object is generated based on the inverse tangent of the ratio of the first difference to the second difference. This simplifies the process of generating the heading angle tag by using the inverse tangent function and the distance between the corner points of the first target object.

[0173] The present application embodiment proposes a target detection method. Figure 2 A schematic diagram of the implementation process of a target detection method proposed in an embodiment of the present application is shown in FIG. Figure 2 As shown, the target detection method includes the following steps S201 and S202:

[0174] Step S201: Acquire a second point cloud image to be detected;

[0175] Here, the second point cloud image is a three-dimensional point cloud image collected by a sensor installed on the vehicle. In one example, the sensor may be a laser radar.

[0176] Step S202: Detect the second target object in the second point cloud image using a target detection model to obtain detection information of the second target object; wherein the target detection model is obtained using the above-mentioned model training method.

[0177] Here, the target detection model is the target detection model to be trained, which is based on the corner point labels and the heading angle labels as described in the above embodiment, and is obtained by performing supervised training on the target detection model to be trained.

[0178] In some embodiments, the second point cloud image is input into a target detection model, and the target detection model is used to predict the corner points and heading angles of the second target object in the second point cloud image, and output the corner point coordinates and heading angles of the second target object.

[0179] In some implementations, the heading angle is determined based on corner points of the second target object, which simplifies the calculation process of the heading angle.

[0180] In some embodiments, a target detection model is used to detect the corner points and heading angle of the second target object in the second point cloud image, and corner point detection results and heading angle detection results can be obtained. Based on the corner point detection results and heading angle detection results, as well as pre-set corner point labels and heading angle labels, it can be determined whether the detection prediction results meet the requirements. If the detection results meet the requirements, the corner points and heading angles of the second target object obtained by detection are output; if the detection results do not meet the requirements, the parameters of the target detection model can be updated based on the corner point detection results and heading angle detection results, as well as pre-set corner point labels and heading angle labels, and then the target detection model with updated parameters can be used to re-detect the corner points and heading angle of the second target object until the detection results meet the requirements, and the corner points and heading angles of the second target object are output.

[0181] In the embodiment of the present application, a second point cloud image to be detected is obtained, and the target detection model obtained by the above-described model training method is used to detect the second target object in the second point cloud image to obtain detection information of the second target object. Thus, because the trained target detection model is obtained through supervised training based on the corner point labels and heading angle labels, the trained target detection model can accurately detect the corner points and heading angles of the second target object.

[0182] The following describes the application of the embodiments of the present application in actual scenarios.

[0183] With the rapid development of technologies such as autonomous driving, intelligent robots, and drones, three-dimensional object detection, as a key technology for environmental perception, has become a key focus in the field of computer vision. The goal of 3D object detection is to accurately identify and locate objects from two-dimensional images or three-dimensional point cloud data, and to infer the objects' spatial position, size, and posture (i.e., orientation and angle). With technological advancements, 3D object detection technology has gradually expanded to include data fusion from multiple sensors, including RGB cameras, LiDAR, depth cameras, radar, and more.

[0184] In related technologies, 3D object detection methods mostly rely on laser radar (LiDAR) or depth cameras (RGB-D) to obtain three-dimensional information. However, these 3D object detection methods have the following problems:

[0185] Accuracy Issues: In complex scenes, depth estimation is easily affected by environmental variations (such as lighting, occlusion, and reflections), resulting in large errors in the estimation results. Therefore, extracting high-quality 3D information from 2D images remains an important research topic.

[0186] Multi-sensor fusion: How to effectively fuse data from different sensors and extract valuable spatial information remains a technical challenge. The data formats, accuracy, and sampling frequencies of different sensors vary significantly, requiring the design of complex data fusion algorithms to effectively synthesize 3D spatial information.

[0187] Model complexity and real-time performance: 3D object detection typically involves a large amount of computation, especially when using deep neural networks. This requires processing a large number of feature maps and computing resources, resulting in high computational overhead. This is particularly prominent in applications with high real-time processing requirements (such as autonomous driving and robot navigation).

[0188] Difficulty in estimating target pose (heading angle): The calculation of heading angle usually relies on complex sensor data and high-dimensional geometric models, which makes the estimation process complicated and performs poorly for target objects in unknown environments.

[0189] Although a large number of deep learning-based image processing methods have achieved promising results, such as convolutional neural networks (CNNs) and regional convolutional neural networks (R-CNNs), they still face technical challenges such as how to efficiently and accurately obtain three-dimensional information from 2D images, how to reduce computational overhead, and how to improve the accuracy of object pose estimation. Furthermore, 3D object detection methods in related technologies typically only address the position and size of the object, with less attention paid to estimating the object's pose.

[0190] Based on the above description, the embodiment of the present application proposes a corner point-based 3D target detection model training method, which optimizes the label generation method during model training, can effectively extract the three-dimensional position information of the target object in the two-dimensional image, accurately estimate the target's posture (heading angle), and can improve the positioning accuracy, posture estimation accuracy and computational efficiency of the target object, providing a more reliable and efficient 3D target detection solution for applications such as autonomous driving and robot perception.

[0191] During model training, we first preprocess datasets such as KITTI, nuScenes, and Waymo to generate size labels, corner labels, center point labels, and heading angle labels to obtain a sample training set. These datasets provide rich object annotation data, including the object's 3D bounding box (size information), center point location, heading angle, and other information.

[0192] A corner point is a function of the target object's center position, dimensions, and heading angle. Simple geometric calculations can be used to calculate the real-world location of the corner points. For each target object's 3D bounding box, trigonometric calculations are used to determine the coordinates of the object's four corner points based on its dimensions (length, width, height), center point, and attitude (heading angle). Based on the 3D coordinates of the target object's center point and the rotation matrix (the rotation matrix between the local coordinate system and the world coordinate system, determined by the heading angle), the coordinates of the target object's corner points are converted from the local coordinate system to the world coordinate system. These 3D corner point coordinates are then projected onto the 2D image plane using the camera's intrinsic and extrinsic parameters to obtain accurate corner coordinates.

[0193] Heading refers to the rotation angle of the target object on a two-dimensional plane and is one of the posture information of the target object.

[0194] Generation of corner point labels: By calculating the distance between the center point and the corner point of the target object, and using these distances as corner point labels, the target detection model to be trained can be guided to predict and regress the corner points based on the corner point labels. This not only improves the prediction accuracy of the corner point coordinates of the target object, but also provides accurate center position coordinates for subsequent spatial positioning and tracking tasks. The corner point label design can be as follows: Figure 3 As shown, the rectangular frame of the target object has four corner points, namely corner point 0, corner point 1, corner point 2, corner point 3, and center point 31. According to the distances between corner point 0, corner point 1, corner point 2, corner point 3 and center point 31, the corner point labels corresponding to the four corner points are generated.

[0195] Size label generation: Generate a size label based on the rectangular box of the target object to guide the object detection model to be trained to predict and regress the size based on the size label.

[0196] Generation of heading angle labels: By calculating the relative positions of the corner points of the target object, the precise value of the heading angle is obtained. See formula (6). The calculation of the heading angle is based on the position difference between the two corner points on the two-dimensional image plane, and the inverse tangent function is used to calculate the heading angle of the target object. During the training process, the heading angle is used as the label of the regression task. The heading angle label guides the target detection model to be trained to learn how to predict the heading angle based on the relative positions between the corner points. The heading angle label design can be as follows: Figure 4 As shown, the heading angle can be calculated based on the coordinates of corner point 2 and corner point 3 using the inverse tangent function.

[0197] The heading angle calculation method in related technologies usually relies on complex three-dimensional sensor data, while this application simplifies the calculation of the heading angle through simple corner point geometric relationships, which not only simplifies the calculation process of the heading angle, but also improves the expressiveness of the model in dynamic scenes, enabling the model to accurately capture the target's direction information, thereby improving the accuracy and efficiency of 3D target detection.

[0198] Loss function design: The loss function determines how the target detection model optimizes various tasks of the target object during training, including corner regression, size regression, center point regression, and heading angle estimation. To ensure that the model can accurately regress the corners, size, center point, and heading angle of the target object, the embodiment of this application proposes a composite loss function that comprehensively considers multiple regression tasks. The loss function contains multiple branches, each branch corresponding to a specific task. The following is the detailed design of the loss function:

[0199] (1) Corner regression loss

[0200] The purpose of corner regression is to predict the coordinates of the four corner points of the target object through the target detection model, ensuring that the target detection model can accurately locate the bounding box of the target object. To optimize the accuracy of corner regression, the embodiment of the present application uses Smooth L1 Loss as the regression loss. Smooth L1 Loss behaves as L2 Loss for small errors and L1 Loss for large errors. Therefore, it can avoid the excessive penalty of L2 Loss for large errors and accelerate convergence for small errors. The calculation formula of corner regression loss is shown in Formula (2).

[0201] (2) Dimension regression loss

[0202] To enable the object detection model to accurately regress the size of the target object, this application uses L2 loss to optimize the target object size regression task. L2 loss can effectively reduce the gap between the predicted size and the true size, enabling the object detection model to accurately regress the 3D size of the target object. The calculation formula for the size regression loss is shown in Formula (1).

[0203] (3) Center point regression loss

[0204] In order to enable the target detection model to accurately regress the center point of the target object and thus accurately predict the position of the target object, the embodiment of the present application designs a loss function of the center point, see formula (3).

[0205] (4) Heading angle regression loss

[0206] The heading angle is an important indicator of the target's posture. Especially in dynamic target detection, the accuracy of the heading angle estimation directly affects the target's spatial positioning and subsequent tasks (such as path planning). To ensure accurate heading angle regression, this application uses a smooth L1 loss to optimize the heading angle regression. The calculation formula for the heading angle regression loss is shown in Formula (4).

[0207] (5) Total loss function

[0208] To comprehensively optimize corner regression, size regression, center point regression, and heading angle regression, this application designs a weighted composite loss function. Specifically, the four loss functions (corner regression loss, size regression loss, center point regression loss, and heading angle regression loss) are summed according to certain weights to obtain the final training loss. The formula for calculating the total loss function is shown in Formula (5).

[0209] In the embodiments of this application, corner point labels are generated based on the relative distance between corner points and the center point, enabling the model to more accurately regress the target's corner point coordinates, thereby accurately predicting the target object's center point coordinates. This provides more precise spatial information for downstream tasks (such as target tracking and path planning), effectively improving target positioning accuracy, especially in dynamic scenarios. Furthermore, the geometric relationship between corner points has been redesigned to simplify the complex heading angle calculation process. This not only reduces the complexity of heading angle calculation but also improves the accuracy of target direction estimation, offering significant advantages for accurate attitude estimation of dynamic targets.

[0210] Based on the above embodiments, the present invention provides a model training device. Figure 5 This is a schematic diagram of the structure of a model training device proposed in an embodiment of the present application, as shown in FIG. Figure 5As shown, the model training device 500 includes: a first acquisition module 501, used to acquire a first point cloud image containing a first target object; the first point cloud image is obtained by vehicle acquisition, and the first target object has label information, and the label information includes the corner point label of the first target object and the heading angle label of the first target object; the corner point label of the first target object is generated based on the distance between the corner point of the first target object and the center point of the first target object; a first detection module 502, used to use the target detection model to be trained to detect the first target object in the first point cloud image to obtain detection information of the first target object, and the detection information of the first target object includes the corner point prediction value of the first target object and the heading angle prediction value of the first target object; a first determination module 503, used to determine the target loss value based on the label information of the first target object and the detection information of the first target object; a parameter updating module 504, used to perform at least one parameter update on the target detection model to be trained based on the target loss value to obtain a trained target detection model.

[0211] In some embodiments, the label information also includes a size label of the first target object and a center point label of the first target object, and the detection information of the first target object also includes a size prediction value of the first target object and a center point prediction value of the first target object; the above-mentioned first determination module includes: a first determination unit, used to determine the size regression loss based on the size prediction value of the first target object and the size label of the first target object; a second determination unit, used to determine the corner regression loss based on the corner prediction value of the first target object and the corner label of the first target object; a third determination unit, used to determine the center point regression loss based on the center point prediction value of the first target object and the center point label of the first target object; a fourth determination module, used to determine the heading angle regression loss based on the heading angle prediction value of the first target object and the heading angle label of the first target object; a fifth determination unit, used to determine the target loss value based on the size regression loss, the corner regression loss, the center point regression loss and the heading angle regression loss.

[0212] In some embodiments, the above-mentioned fifth determination unit includes: a first determination subunit, used to respectively determine the first weight coefficient of the size regression loss, the second weight coefficient of the corner point regression loss, the third weight coefficient of the center point regression loss and the fourth weight coefficient of the heading angle regression loss; a second determination subunit, used to determine the target loss value based on the first weight coefficient, the second weight coefficient, the third weight coefficient, the fourth weight coefficient, the size regression loss, the corner point regression loss, the center point regression loss and the heading angle regression loss.

[0213] In some embodiments, the above-mentioned model training device also includes: a second determination module, used to determine the corner points of the first target object based on the size information of the first target object, the center point of the first target object and the heading angle of the first target object; a third determination module, used to determine the corner point labels of the first target object based on the corner points of the first target object and the center point of the first target object.

[0214] In some embodiments, the above-mentioned second determination module includes: a sixth determination unit, used to determine the corner points of the first target object in the local coordinate system based on the size information of the first target object and the center point of the first target object; the local coordinate system is a coordinate system with the center point of the first target object as the coordinate origin; a seventh determination unit, used to determine the rotation matrix based on the heading angle; a rotation and translation unit, used to rotate and translate the corner points of the first target object in the local coordinate system based on the rotation matrix to obtain the corner points of the first target object in the world coordinate system.

[0215] In some embodiments, the corner points of the first target object include a first corner point and a second corner point, and the above-mentioned model training device also includes: a fourth determination module, used to determine the first difference between the horizontal coordinate of the first corner point and the horizontal coordinate of the second corner point; a fifth determination module, used to determine the second difference between the vertical coordinate of the first corner point and the vertical coordinate of the second corner point; and a generation module, used to generate a heading angle label of the first target object based on the first difference and the second difference.

[0216] In some embodiments, the generation module includes: an eighth determination unit for determining a ratio between the first difference and the second difference; and a generation unit for generating a heading angle label of the first target object based on an arctangent value of the ratio.

[0217] The present invention provides a target detection device. Figure 6 This is a schematic diagram of the structure of a target detection device proposed in an embodiment of the present application, as shown in FIG. Figure 6 As shown, the target detection device 600 includes: a second acquisition module 601, used to acquire a second point cloud image to be detected; a second detection module 602, used to use a target detection model to detect a second target object in the second point cloud image to obtain detection information of the second target object; wherein, the target detection model is obtained using the above-mentioned model training method.

[0218] The description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to perform the methods described in the above method embodiments. For technical details not disclosed in the device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0219] This embodiment further proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements some or all of the steps in the above method when executing the program.

[0220] This embodiment further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements some or all of the steps in the above method. The computer-readable storage medium may be transient or non-transient.

[0221] This embodiment further provides a computer program, including computer-readable codes. When the computer-readable codes are run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0222] This embodiment also provides a computer program product, comprising a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, the computer program implements some or all of the steps of the above method. The computer program product can be implemented through hardware, software, or a combination thereof. In some embodiments, the computer program product is embodied as a computer storage medium. In other embodiments, the computer program product is embodied as a software product, such as a software development kit (SDK).

[0223] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between the various embodiments, and their similarities or similarities can be referenced to each other. The descriptions of the above device, storage medium, computer program, and computer program product embodiments are similar to the descriptions of the above method embodiments and have similar beneficial effects as the method embodiments. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the description of the method embodiments of this application for understanding.

[0224] It should be noted that the embodiment of the present application provides a hardware entity of a computer device, such as Figure 7As shown, the hardware components of computer device 700 include a processor 701, which generally controls the overall operation of computer device 700. A communication interface 702 enables the computer device to communicate with other terminals or servers via a network. A memory 703 is configured to store instructions and applications executable by processor 701 and to cache data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or already processed by processor 701 and various modules within computer device 700. This can be implemented using flash memory (FLASH) or random access memory (RAM). Data can be transmitted between processor 701, communication interface 702, and memory 703 via bus 704.

[0225] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned steps / processes does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0226] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A model training method, characterized in that: include: Acquire a first point cloud image containing a first target object; The first point cloud image is acquired by vehicle acquisition, the first target object has label information, the label information includes corner point labels of the first target object and a heading angle label of the first target object, the corner point labels of the first target object are generated based on distances between corner points of the first target object and a center point of the first target object, the corner points of the first target object are determined based on size information of the first target object, the center point of the first target object, and the heading angle of the first target object; and the heading angle label of the first target object is generated based on relative positions between the corner points of the first target object; Detecting a first target object in the first point cloud image using the target detection model to be trained to obtain detection information of the first target object, where the detection information of the first target object includes a predicted value of a corner point of the first target object and a predicted value of a heading angle of the first target object; determining a target loss value based on the label information of the first target object and the detection information of the first target object; Based on the target loss value, the parameters of the target detection model to be trained are updated at least once to obtain a trained target detection model.

2. The method according to claim 1, characterized in that The label information further includes a size label of the first target object and a center point label of the first target object, and the detection information of the first target object further includes a size prediction value of the first target object and a center point prediction value of the first target object; The determining the target loss value based on the label information of the first target object and the detection information of the first target object includes: determining a size regression loss based on the size prediction value of the first target object and the size label of the first target object; Determining a corner regression loss based on the corner prediction value of the first target object and the corner label of the first target object; Determining a center point regression loss based on the center point prediction value of the first target object and the center point label of the first target object; Determining a heading angle regression loss based on the heading angle prediction value of the first target object and the heading angle label of the first target object; The target loss value is determined based on the size regression loss, the corner point regression loss, the center point regression loss, and the heading angle regression loss.

3. The method according to claim 2, characterized in that The determining the target loss value based on the size regression loss, the corner point regression loss, the center point regression loss, and the heading angle regression loss includes: Respectively determining a first weight coefficient of the size regression loss, a second weight coefficient of the corner point regression loss, a third weight coefficient of the center point regression loss, and a fourth weight coefficient of the heading angle regression loss; The target loss value is determined based on the first weight coefficient, the second weight coefficient, the third weight coefficient, the fourth weight coefficient, the size regression loss, the corner point regression loss, the center point regression loss and the heading angle regression loss.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: determining a corner point of the first target object based on size information of the first target object, a center point of the first target object, and a heading angle of the first target object; Determine corner point labels of the first target object based on the corner points of the first target object and the center point of the first target object.

5. The method according to claim 4, characterized in that The determining the corner points of the first target object based on the size information of the first target object, the center point of the first target object, and the heading angle of the first target object includes: Determining, based on the size information of the first target object and the center point of the first target object, the corner points of the first target object in a local coordinate system; the local coordinate system is a coordinate system with the center point of the first target object as a coordinate origin; determining a rotation matrix based on the heading angle; Based on the rotation matrix, the corner points of the first target object in the local coordinate system are rotated and translated to obtain the corner points of the first target object in the world coordinate system.

6. The method according to any one of claims 1 to 3, characterized in that The corner points of the first target object include a first corner point and a second corner point, and the method further includes: determining a first difference between the abscissa of the first corner point and the abscissa of the second corner point; determining a second difference between the longitudinal coordinate of the first corner point and the longitudinal coordinate of the second corner point; A heading angle label of the first target object is generated based on the first difference and the second difference.

7. The method according to claim 6, characterized in that Generating a heading angle label of the first target object based on the first difference and the second difference includes: determining a ratio between the first difference and the second difference; A heading angle label of the first target object is generated based on an arctangent value of the ratio.

8. A target detection method, characterized in that: include: Acquire a second point cloud image to be detected; Use the target detection model to detect the second target object in the second point cloud image to obtain detection information of the second target object; wherein, the target detection model is obtained using the model training method described in any one of claims 1 to 7.

9. A model training device, characterized in that: include: A first acquisition module is used to acquire a first point cloud image containing a first target object; The first point cloud image is acquired by vehicle acquisition, the first target object has label information, the label information includes corner point labels of the first target object and a heading angle label of the first target object, the corner point labels of the first target object are generated based on distances between corner points of the first target object and a center point of the first target object, the corner points of the first target object are determined based on size information of the first target object, the center point of the first target object, and the heading angle of the first target object; and the heading angle label of the first target object is generated based on relative positions between the corner points of the first target object; a first detection module, configured to detect a first target object in the first point cloud image using a target detection model to be trained, and obtain detection information of the first target object, where the detection information of the first target object includes a predicted value of a corner point of the first target object and a predicted value of a heading angle of the first target object; a first determining module, configured to determine a target loss value based on the label information of the first target object and the detection information of the first target object; A parameter updating module is used to update the parameters of the target detection model to be trained at least once based on the target loss value to obtain a trained target detection model.

10. A target detection device, characterized in that: include: A second acquisition module is used to acquire a second point cloud image to be detected; A second detection module is used to detect a second target object in the second point cloud image using a target detection model to obtain detection information of the second target object; wherein the target detection model is obtained using the model training method described in any one of claims 1 to 7.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A vehicle, characterized in that: Comprising the computer device of claim 11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images