Model training, target detection, vehicle control method, device, equipment and medium

Abstract

This disclosure provides a model training, object detection, and vehicle control method, apparatus, device, and medium, relating to the field of artificial intelligence, specifically to intelligent transportation and autonomous driving. The specific implementation scheme includes: acquiring training samples; rotating the sample environment point cloud in a first manner to obtain a rotated sample environment point cloud; extracting input data of the target size from the rotated sample environment point cloud and inputting the input data into an object detection model to obtain the detection result output by the object detection model; the target size corresponds to the target detection range of the object detection model, and the target detection range has different detection intervals in different directions; adjusting the parameters of the object detection model based on the difference between the detection result output by the object detection model and each of the ground truth boxes. Embodiments of this disclosure can improve the model detection speed and accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically to the fields of intelligent transportation and autonomous driving, and particularly to a method, apparatus, device, and medium for model training, target detection, and vehicle control. Background Technology

[0002] During the operation of an autonomous vehicle, the autonomous driving perception system needs to accurately detect and identify surrounding obstacles in order to take timely avoidance measures and prevent traffic accidents from occurring.

[0003] LiDAR (Light Detection and Ranging) acquires three-dimensional point cloud data of the surrounding environment by emitting laser beams and measuring the time it takes for them to reflect back. As a core sensor for autonomous driving, LiDAR data offers accurate ranging and is highly adaptable to environmental conditions, providing accurate environmental information to autonomous driving systems. Summary of the Invention

[0004] This disclosure provides a model training, target detection, vehicle control method, apparatus, device, and medium.

[0005] According to one aspect of this disclosure, a method for training an object detection model is provided, comprising:

[0006] Acquire training samples, which include a sample environment point cloud of the target device and at least one ground truth bounding box;

[0007] The sample environment point cloud is rotated in a first manner to obtain a rotated sample environment point cloud, so that the point cloud in the sample environment point cloud at the position of the preset movement direction of the target device is rotated to the position of the preset detection direction.

[0008] Input data of the target size is extracted from the rotated sample environment point cloud and input into the target detection model to obtain the detection result output by the target detection model; the target size corresponds to the target detection range of the target detection model, the target detection range has different detection intervals in different directions, and the target detection range includes the detection interval corresponding to the preset detection direction;

[0009] The parameters of the target detection model are adjusted based on the differences between the detection results output by the target detection model and each of the ground truth boxes.

[0010] According to another aspect of this disclosure, a target detection method is provided, comprising:

[0011] Obtain the vehicle's environmental point cloud;

[0012] The vehicle environment point cloud is rotated in a first manner to obtain a rotated vehicle environment point cloud, so that the point cloud in the vehicle environment point cloud at the position of the vehicle's preset movement direction is rotated to the position of the preset detection direction.

[0013] Input data of the target size is extracted from the rotated vehicle environment point cloud and input into a pre-trained target detection model to obtain the detection result output by the target detection model; the target size corresponds to the target detection range of the target detection model, and the target detection range has different detection intervals in different directions, including the detection interval corresponding to the preset detection direction;

[0014] The target bounding box in the detection result output by the target detection model is rotated in a second way to obtain the target detection result. The second way is the inverse of the first way.

[0015] According to another aspect of this disclosure, a vehicle control method is provided, comprising:

[0016] The target detection result of the vehicle is obtained, and the target detection result is obtained by the target detection method described in any embodiment of this disclosure;

[0017] Based on the target detection results, the vehicle is controlled to move.

[0018] According to one aspect of this disclosure, a training apparatus for an object detection model is provided, comprising:

[0019] The training sample input acquisition module is used to acquire training samples, which include a sample environment point cloud of the target device and at least one truth box;

[0020] The point cloud rotation module is used to rotate the sample environment point cloud in a first manner to obtain the rotated sample environment point cloud, so that the point cloud in the sample environment point cloud at the position of the preset movement direction of the target device is rotated to the position of the preset detection direction.

[0021] An input cropping module is used to crop the input data of the target size from the rotated sample environment point cloud, and input the input data into the target detection model to obtain the detection result output by the target detection model; the target size corresponds to the target detection range of the target detection model, the target detection range has different detection intervals in different directions, and the target detection range includes the detection interval corresponding to the preset detection direction;

[0022] The model training module is used to adjust the parameters of the target detection model based on the differences between the detection results output by the target detection model and each of the ground truth boxes.

[0023] According to another aspect of this disclosure, a target detection apparatus is provided, comprising:

[0024] The point cloud data acquisition module is used to acquire the vehicle's environmental point cloud.

[0025] The point cloud data rotation module is used to rotate the vehicle environment point cloud in a first manner to obtain the rotated vehicle environment point cloud, so that the point cloud in the vehicle environment point cloud at the position of the vehicle's preset movement direction is rotated to the position of the preset detection direction.

[0026] The input data extraction module is used to extract the target size input data from the rotated vehicle environment point cloud and input it into the pre-trained target detection model to obtain the detection result output by the target detection model; the target size corresponds to the target detection range of the target detection model, the target detection range has different detection intervals in different directions, and the target detection range includes the detection interval corresponding to the preset detection direction;

[0027] The detection result rotation module is used to rotate the target box in the detection result output by the target detection model in a second way to obtain the target detection result. The second way is the inverse of the first way.

[0028] According to another aspect of this disclosure, a vehicle control device is provided, comprising:

[0029] The target detection module is used to acquire the target detection results of the vehicle, wherein the target detection results are acquired by the target detection method described in any embodiment of this disclosure;

[0030] The vehicle control module is used to control the vehicle's movement based on the target detection results.

[0031] According to another aspect of this disclosure, a training apparatus for an object detection model is provided, comprising:

[0032] At least one processor; and

[0033] A memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the training method of the object detection model according to any embodiment of this disclosure.

[0035] According to another aspect of this disclosure, a target detection device is provided, comprising:

[0036] At least one processor; and

[0037] A memory communicatively connected to the at least one processor; wherein,

[0038] The memory stores instructions that can be executed by the at least one processor, which, when executed, enable the at least one processor to perform the target detection method according to any embodiment of this disclosure.

[0039] According to another aspect of this disclosure, a vehicle control device is provided, comprising:

[0040] At least one processor; and

[0041] A memory communicatively connected to the at least one processor; wherein,

[0042] The memory stores instructions that can be executed by the at least one processor, which, when executed, enables the at least one processor to perform the vehicle control method according to any embodiment of this disclosure.

[0043] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform a training method or a target detection method of the target detection model described in any embodiment of this disclosure.

[0044] According to another aspect of this disclosure, an autonomous vehicle is provided, including: the target detection device and / or vehicle control device described in any embodiment of this disclosure.

[0045] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a training method or object detection method for an object detection model according to any embodiment of this disclosure.

[0046] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0047] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0048] Figure 1 This is a flowchart of a training method for an object detection model disclosed in an embodiment of this disclosure;

[0049] Figure 2 This is a schematic diagram of a sample environment point cloud before and after rotation, according to an embodiment of the present disclosure;

[0050] Figure 3 This is a flowchart of another training method for an object detection model disclosed in an embodiment of this disclosure;

[0051] Figure 4 This is a schematic diagram of a method for matching anchor frames and truth frames before and after rotation, according to an embodiment of this disclosure.

[0052] Figure 5 This is a flowchart of a target detection method disclosed in an embodiment of the present disclosure;

[0053] Figure 6 This is a flowchart of a vehicle control method disclosed in an embodiment of the present disclosure;

[0054] Figure 7 This is a schematic diagram of the structure of a training device for the target detection model disclosed in an embodiment of this disclosure;

[0055] Figure 8 This is a schematic diagram of the target detection device disclosed in the embodiments of this disclosure;

[0056] Figure 9 This is a schematic diagram of the structure of a vehicle control device disclosed according to an embodiment of the present disclosure;

[0057] Figure 10 This is a block diagram of a training method for an object detection model or an electronic device for object detection disclosed in embodiments of this disclosure. Detailed Implementation

[0058] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0059] Figure 1 This is a flowchart of a training method for an object detection model according to an embodiment of the present disclosure. This embodiment can be applied to training an object detection model for detecting obstacles around a vehicle. The method of this embodiment can be executed by a training device for the object detection model. This device can be implemented in software and / or hardware and is specifically configured in an electronic device with a certain data processing capability. The electronic device can be a client device or a server device. Client devices include mobile phones, tablets, vehicle terminals, and desktop computers.

[0060] S101. Obtain training samples, wherein the training samples include the sample environment point cloud of the target device and at least one ground truth box.

[0061] A large number of training samples can be collected to train the object detection model. For example, the object detection model can be a deep learning model that uses anchor boxes for object detection. For example, the object detection model can be Fast R-CNN (Fast Region-based Convolutional Network), SDD (Single Shot MultiBox Detector), or YOLO (You Only Look Once), etc.

[0062] Training samples include input data and labeled data. The input data is a sample environment point cloud. The labeled data are ground truth boxes. The sample environment point cloud can contain at least one object, and correspondingly, there can be at least one ground truth box. The sample environment point cloud can be point cloud data of the environment to be detected, specifically a point cloud collected by an acquisition device, or a point cloud obtained by processing a point cloud collected by an acquisition device. For example, a point cloud collected by radar is acquired and converted to a bird's-eye view to obtain a two-dimensional point cloud, which serves as the sample environment point cloud. The bird's-eye view (BEV) can be the perspective of looking down at the ground from the air. The acquisition device can be configured on a target device, which may include a sample vehicle or robot; the robot may include a food delivery robot or a logistics robot, etc. A ground truth box refers to the space or region of the object to be detected in the sample environment point cloud. Ground truth boxes can be represented by coordinates, such as the coordinates of the vertex of the ground truth box, or the coordinates of the center point of the ground truth box and the distance between the boundary, or the coordinates of the center point of the ground truth box and the distance between the boundary and the center point, etc. In addition, the truth box can be labeled with the category of the object to be detected. For example, the category can include static objects or dynamic objects. It can be further subdivided; for example, dynamic objects include pedestrians, small animals, or vehicles. Static objects can include signs, railings, flower beds, dining tables, shelves, or trash cans. This can be configured according to the specific scenario.

[0063] S102. Rotate the sample environment point cloud in a first manner to obtain a rotated sample environment point cloud, so that the point cloud in the sample environment point cloud at the position of the preset movement direction of the target device is rotated to the position of the preset detection direction.

[0064] The first method can be the rotation direction and angle of the point cloud. In practice, the target device can be equipped with multiple viewing angle acquisition devices to collect point clouds of the environment within a wide angle centered on the target device; for example, collecting point clouds of the environment surrounding the target device in a 360-degree radius. The first method of rotation can be rotating the sample environment point cloud centered on the target device, specifically rotating it clockwise or counterclockwise. The preset movement direction of the target device can refer to the current direction of movement. For example, it can be directly in front of the target device; for instance, directly in front could be the direction the front of a vehicle is facing, or the direction the robot is facing. For example, if the target device moves due north, directly in front of the target device is due north, and the preset movement direction can be due north. Alternatively, directly in front can be a range of ±45 degrees in a coordinate system with the target device's positive x-axis; the preset movement direction can be the direction corresponding to this ±45-degree range in the x-axis coordinate system. Furthermore, other situations exist and can be set as needed; these are not specifically limited.

[0065] For each point cloud in the sample environment, a first rotation is performed: specifically, the point cloud rotates from a position in one direction to a position in another direction, while the distance between the point cloud and the target device remains unchanged before and after the rotation. The point cloud in the sample environment that is located at a position in a preset movement direction of the target device is positioned in that preset movement direction before rotation, and then positioned in a preset detection direction after rotation. The preset detection direction can be a direction selected from multiple detection intervals of the target detection model.

[0066] In practice, the detection interval of an object detection model differs in different directions. The direction of the object detection model can refer to the direction from the detection center point to the boundary points of the detection range. The detection interval can be the range formed by line segments from the detection center point to the boundary points of the detection range. The detection center point is usually the center point of the target detection range; for example, if the target detection range is circular, the center point is the center of the circle; or if the target detection range is square, the center point is the intersection of the diagonals. Alternatively, the vector set formed by the detection center point and multiple consecutive adjacent boundary points of the detection range can be defined as a direction, and the corresponding detection interval is the range formed by multiple line segments from the detection center point to these consecutive adjacent boundary points. The difference in detection intervals is specifically reflected in their range and / or length. The detection interval length is the same in some detection directions and different in others.

[0067] The detection interval corresponding to the preset movement direction can be obtained and compared with the detection intervals of each detection direction of the target detection model. A certain detection direction is selected as the preset detection direction. The point cloud at the position of the preset movement direction is rotated with the target device as the center to the corresponding position in the preset detection direction, so that the preset movement direction of the target device is aligned with the preset detection direction. Here, alignment can mean that the vector representing the preset movement direction and the vector representing the preset detection direction are similar. For example, alignment can mean that the angle between them is less than a preset angle threshold. For example, a detection interval longer than the detection interval of the detection direction corresponding to the preset movement direction can be selected, and the corresponding detection direction is the preset detection direction. This can make more point clouds at the preset movement direction fall within the detectable interval, thereby achieving the detection of more objects in a wider point cloud.

[0068] S103. Extract the target size input data from the rotated sample environment point cloud and input the input data into the target detection model to obtain the detection result output by the target detection model; the target size corresponds to the target detection range of the target detection model, the target detection range has different detection intervals in different directions, and the target detection range includes the detection interval corresponding to the preset detection direction.

[0069] The target size can refer to the size of the space defined by the target detection range corresponding to the target detection model. The input data for extracting the target size from the rotated sample environment point cloud can be: defining the space of the target size within the sample environment point cloud centered on the target device, and using the point cloud within that target size space as the input data. The target detection model processes the input data to obtain the detection results output by the target detection model. The detection results are the predicted bounding boxes of sample objects in the rotated sample environment point cloud, where the predicted bounding boxes can be represented by the coordinates of key points and / or the box size. The detection results may also include: the predicted category of the sample objects contained within the predicted bounding boxes.

[0070] The target detection range can be considered as the detectable space in a point cloud target detection operation, where the detectable space represents the space with the highest detection accuracy. The target detection range in a specific detection direction can be understood as the farthest object that the target device can detect in that direction. In practice, the input data for a target detection model is typically a regularly shaped spatial data set. Simultaneously, the target device is actually located at the center of the space to be detected, and the target device has a direction of movement and orientation. Therefore, an directional coordinate system can be established based on the target device's direction of movement or orientation. Consequently, the target detection model has different detection intervals in different directions within the directional coordinate system of the target device; these different detection intervals constitute the target detection range of the target detection model.

[0071] Typically, the distribution range of the sample environment point cloud is larger than the target detection range of the object detection model. When inputting the sample environment point cloud into the object detection model, the size corresponding to the target detection range is usually truncated before being input into the object detection model for detection. Optionally, the detection interval corresponding to the aforementioned preset detection direction can be the longest detection interval among the detection intervals corresponding to each direction included in the target detection range.

[0072] like Figure 2 As shown, in the scenario where the vehicle is moving in the direction of the arrow, Figure 2 The grid area shown is the detection range in the direction of the arrow (i.e., the preset movement direction). The target detection range is a 120*120 rectangle, and the distribution range of the sample environment point cloud is the outermost rectangle. Figure 2 It can be seen that before rotation, the point cloud directly in front of the vehicle within the dashed circle is outside the target detection range (a 120*120 rectangle) and inside a circle of 160 degrees (the diagonal length of the 120*120 rectangle is 120*√2). After rotating counterclockwise by 45 degrees, the point cloud directly in front of the vehicle within the dashed circle is now within the target detection range (a 120*120 rectangle), and thus falls within the target detection range. Therefore, after rotation, the point cloud directly in front of the vehicle in the direction of the arrow moves into the detectable range, increasing the perception distance of obstacles directly in front of the vehicle.

[0073] In a specific example, the object detection model extracts features from the input data, specifically by downsampling by a factor of 2, to obtain a first feature map. Based on this first feature map, a first anchor box is used to detect small objects. Further feature extraction is then performed on the first feature map, again by downsampling by a factor of 2, to obtain a second feature map. Based on this second feature map, a second anchor box is used to detect large objects. The first and second anchor boxes have different sizes; typically, the second anchor box is larger than the first. For example, in autonomous driving applications, small objects include pedestrians, traffic cones, and trash cans. Large objects include non-motorized vehicles and motorized vehicles.

[0074] S104. Adjust the parameters of the target detection model based on the difference between the detection results output by the target detection model and each of the ground truth boxes.

[0075] The loss value is calculated based on the difference between the detected result and the ground truth box. The parameters of the object detection model are adjusted with the goal of minimizing or converging the loss value. Once trained, the object detection model can be used to detect objects. This model can simultaneously detect objects of multiple categories.

[0076] Optionally, the detection interval corresponding to the preset detection direction includes: the longest detection interval.

[0077] The detection interval corresponding to the preset detection direction is configured as the longest detection interval, so that the point cloud in the preset movement direction is within the longest detection interval. Consequently, the target device has the longest detection distance in the preset movement direction, so the preset movement direction of the target device can cover obstacles at a greater distance.

[0078] Typically, among at least one movable direction of the target device, the currently moving direction is selected as the preset moving direction. For a target device moving along the preset moving direction, obstacles in the preset moving direction have the greatest impact on the safety of the target device's movement.

[0079] By configuring the detection interval corresponding to the preset detection direction as the longest detection interval, the detection distance of the preset movement direction can be increased, the detection range of the preset movement direction can be expanded, and thus the movement safety of the preset movement direction can be improved.

[0080] Optionally, the target detection range of the target detection model is a square, the preset movement direction before rotation is perpendicular to the edge of the square, and the preset detection direction is the detection direction corresponding to the diagonal of the square; the first method includes rotating the target device as the center along a first direction by a preset angle so that the preset movement direction after rotation is aligned with the detection direction corresponding to the diagonal of the square.

[0081] Typically, the preset movement direction is directly in front of the target device, with the target device at the center of the target detection range. The front direction is perpendicular to the edge of a square, meaning the detection range in front is half the length of the square's edge. For a square, the diagonal is the longest, meaning the detection interval corresponding to the diagonal is the longest. Therefore, the preset detection direction is the direction corresponding to the diagonal. The preset angle can be 45 + n * 90 degrees. Rotating the preset movement direction by 45 + n * 90 degrees aligns it with the direction corresponding to the diagonal of the square, maximizing the detection distance and coverage of the preset movement direction.

[0082] Specifically, with the target device as the center, rotate the sample environment point cloud clockwise or counterclockwise at an angle of 45+n*90, so that the point cloud in the sample environment point cloud in the preset movement direction is in the direction corresponding to the diagonal of the square.

[0083] For applications where the target detection area is a square and the preset movement direction is perpendicular to the edge of the target detection area, rotating the preset movement direction so that it aligns with the diagonal of the square can maximize the detection range of the preset movement direction, increase the detection range of the preset movement direction, and reduce the complexity of increasing the detection range.

[0084] In the field of autonomous driving, LiDAR (Light Detection and Ranging) acquires 3D point cloud data of the surrounding environment by emitting laser beams and measuring the time it takes for them to reflect back. As a core sensor for autonomous driving, LiDAR data offers accurate ranging and is highly adaptable to environmental conditions, providing accurate environmental information to the autonomous driving system. However, considering the safety and reliability of LiDAR, as well as its impact on the surrounding environment and people, its output power has a certain upper limit. This results in the detection range of most LiDAR systems being around 120 meters. Beyond this area, the echo received by the LiDAR sensor decreases, meaning that the point cloud of the scene beyond 120 meters is quite sparse. Therefore, the target detection range is typically set to a 120-meter cube centered on the autonomous vehicle.

[0085] Expanding the detection range can cover more point clouds. However, the marginal benefit of blindly increasing the detection range is obvious. Most of the increased range is empty, and it will also bring a significant increase in latency. This will increase driving risks for autonomous driving, which has extremely high real-time requirements.

[0086] Existing autonomous driving systems typically utilize industrial cameras of varying focal lengths as auxiliary sensors for LiDAR, with telephoto industrial cameras being particularly helpful in improving the detection range of autonomous driving systems. Based on the pinhole imaging principle, telephoto cameras can capture scene information from a greater distance, making them suitable for long-distance detection scenarios such as highways. Using images acquired by telephoto cameras and deep learning detection algorithms, autonomous driving systems can perceive obstacles at a distance.

[0087] Unlike the precise ranging capabilities of LiDAR, image-based detection often suffers from inaccurate depth information. The same detection bounding box may shift within a short period, potentially leading to false speed readings and interfering with autonomous driving decision-making and planning systems, thus posing safety risks. Furthermore, it also has drawbacks such as a smaller field of view, greater difficulty in calibration and debugging, and higher cost.

[0088] According to the technical solution of this disclosure, by rotating the sample environment point cloud in a first manner to align the detection direction of the detection interval corresponding to the preset movement direction with the preset detection direction, the corresponding detection interval can be adjusted for the adaptability of the movement direction, thereby flexibly adjusting the detection range in the movement direction to increase the detection range of the target device in the movement direction, realizing low-cost perception of more distant obstacles, reducing the complexity of detecting distant obstacles, and quickly enabling the target device to detect more distant obstacles, thereby improving the movement safety of the target device.

[0089] Figure 3This is a flowchart of another training method for an object detection model disclosed in this embodiment, which is further optimized and extended based on the above technical solution and can be combined with the above optional implementation methods. The input data is input into the object detection model to obtain the detection result output by the object detection model, specifically as follows: The input data is input into the object detection model; the input data is processed by the object detection model to obtain a feature map; the ground truth boxes are rotated using the first method to obtain at least one rotated ground truth box; multiple standard anchor boxes are generated on the feature map by the object detection model and rotated using the first method to obtain multiple rotated anchor boxes; the rotated anchor boxes are filtered according to the difference between the image data of each rotated anchor box in the feature map and each rotated ground truth box, obtaining a filtering result corresponding to each rotated ground truth box; the predicted box corresponding to each ground truth box and the predicted category corresponding to each predicted box are determined by the object detection model based on the filtering result corresponding to each rotated ground truth box, and are used as the detection result output by the object detection model.

[0090] S301. Obtain training samples, wherein the training samples include the sample environment point cloud of the target device and at least one ground truth box.

[0091] S302. Rotate the sample environment point cloud in a first manner to obtain a rotated sample environment point cloud, so that the point cloud in the sample environment point cloud at the position of the preset movement direction of the target device is rotated to the position of the preset detection direction.

[0092] Specifically, the point cloud collected by the acquisition device on the target device is mapped to a bird's-eye view to obtain a two-dimensional point cloud, which serves as the sample environment point cloud.

[0093] S303. Extract the target size input data from the rotated sample environment point cloud and input the input data into the target detection model; the target size corresponds to the target detection range of the target detection model, and the target detection range has different detection intervals in different directions.

[0094] S304. Using the target detection model, multiple standard anchor frames are generated on the input data and rotated in the first manner to obtain multiple rotated anchor frames.

[0095] Multiple standard anchor boxes with a preset distribution can be generated on the input data. These standard anchor boxes can be upright rectangles or rectangular prisms. Typically, the feature map is a rectangle or rectangular prism, and the standard anchor boxes can be rectangles or rectangular prisms with edges parallel to the corresponding edges of the feature map. For example, the top edge of the standard anchor box is parallel to the top edge of the feature map, the bottom edge is parallel to the bottom edge of the feature map, the left edge is parallel to the left edge of the feature map, and the right edge is parallel to the right edge of the feature map.

[0096] Specifically, taking the center point of the target device or input data as the center, and rotating each standard anchor frame in the same direction as the sample environment point cloud by the same rotation angle, the rotating anchor frame corresponding to each standard anchor frame is obtained.

[0097] S305. The input data is processed by the target detection model to determine the prediction and screening results of each rotating anchor frame, as well as the regression prediction results and category prediction results of the rotating anchor frames that predict positive samples, and these are used as the detection results output by the target detection model.

[0098] The prediction filtering result can refer to the prediction result of whether the rotated anchor box is a positive sample, a negative sample, or an ignored sample. The prediction filtering result can include predicting positive samples, predicting negative samples, and predicting ignored samples. A rotated anchor box predicted as a positive sample indicates that there is a foreground object in the region where the rotated anchor box is located; specifically, it is similar to a rotated ground truth box, meaning the positive sample's rotated anchor box is close to that rotated ground truth box. A rotated anchor box predicted as a negative sample indicates that the region where the rotated anchor box is located is a processable ambiguous sample; ambiguity specifically means that it is impossible to determine whether it is a foreground object or background. Ignored rotated anchor boxes are samples that affect model training.

[0099] Regression prediction results can refer to the results obtained by adjusting the rotated anchor box of the positive sample relative to the rotated truth box. The regression prediction result can be a rotated prediction box obtained by adjusting the rotated anchor box based on the offset between the position and size of the positive sample's rotated anchor box and the position and size of the rotated truth box, or it can be directly represented by the offset between the position and size of the positive sample's rotated anchor box and the position and size of the rotated truth box. Category prediction results can refer to predicting the object category of the rotated anchor box of the positive sample.

[0100] S306. Using the target detection model, rotate each of the truth boxes using the first method to obtain at least one rotated truth box.

[0101] The ground truth boxes are rotated using the same rotation method, and the rotated ground truth boxes are determined as rotated ground truth boxes. In fact, the ground truth boxes in the training samples are rectangular boxes or rectangles on the original sample environment point cloud. Manual annotation of the original sample environment point cloud is unnecessary; the rotated ground truth boxes can be obtained directly from the annotated ground truth boxes on the original sample environment point cloud. Specifically, with the center point of the target device or input data as the center, each ground truth box is rotated at the same rotation angle as the sample environment point cloud, along the same direction as the sample environment point cloud, to obtain the corresponding rotated ground truth boxes.

[0102] S307. Using the target detection model, each rotating anchor frame is filtered based on the difference between each rotating anchor frame and each rotating truth frame to obtain the truth filtering result of each rotating anchor frame.

[0103] The truth filtering result can refer to the correct result of detecting whether a rotated anchor box is a positive sample, a negative sample, or an ignored sample. Each rotated anchor box is compared with each rotated truth box to obtain the difference between them. For each rotated truth box, the corresponding positive sample, negative sample, and ignored sample are determined. Typically, a rotated truth box can correspond to an anchor box with at least one positive sample.

[0104] S308. Using the target detection model, calculate the first difference between the predicted screening result of each of the rotating anchor frames and the true value screening result of each of the rotating anchor frames.

[0105] The first difference is used to determine the classification loss of the rotated anchor boxes. For each rotated anchor box, the difference between the predicted filtering result and the true filtering result of the rotated anchor box for the same rotated truth box is calculated. The difference of the rotated anchor box for each rotated truth box is statistically analyzed, and then the difference of each rotated anchor box is statistically analyzed to obtain the first difference.

[0106] In a specific example, the difference between the predicted filtering result and the true filtering result for the same rotated truth box is x. ij x ij This represents the difference between the predicted filtering result and the true filtering result of the i-th rotated anchor box for the j-th rotated truth box. The differences between this rotated anchor box and each rotated truth box are statistically analyzed. M represents the number of rotated truth boxes. The differences between the rotated anchor boxes are calculated to obtain the first difference. N represents the number of rotating anchor frames.

[0107] S309. Using the target detection model, calculate the second difference between the regression prediction result and the category prediction result of the rotated anchor box that predicts positive samples and the corresponding rotated truth box.

[0108] The second difference is used to determine the regression loss of the rotated anchor frame. The positional and size offsets between the rotated anchor frame and the nearest rotated ground truth frame can be determined based on the regression prediction results of the rotated anchor frame predicting positive samples. Regression and category prediction results are not calculated for rotated anchor frames predicting negative samples or samples to be ignored. The category difference between the category prediction result of the rotated anchor frame predicting positive samples and the category of the rotated ground truth frame is calculated. The regression loss value of the rotated anchor frame predicting positive samples is calculated based on the positional offset, size offset, and category difference. This can be achieved by summing or weighting the positional offset, size offset, and category difference of the rotated anchor frames predicting positive samples. The regression loss values ​​of all the rotated anchor frames predicting positive samples are summed to obtain the second difference.

[0109] S310. Using the target detection model, adjust the parameters of the target detection model based on the first difference and the second difference.

[0110] Calculate the sum or weighted sum of the first and second differences to determine the loss value of the object detection model. The sum or weighted sum of the first and second differences can be used as the loss value of the object detection model. Adjust the parameters of the object detection model with the goal of reducing the loss value or achieving loss convergence until the loss value converges or is minimized. Once training is complete, the object detection model at the current time point is deployed online.

[0111] Optionally, the step of filtering each rotating anchor frame based on the difference between each rotating anchor frame and each rotating truth frame to obtain a truth filtering result for each rotating anchor frame includes: obtaining the truth category of each rotating truth frame; calculating the similarity value between each rotating anchor frame and each rotating truth frame based on the similarity calculation method corresponding to the truth category of each rotating truth frame; and filtering each rotating anchor frame based on the similarity value between each rotating anchor frame and each rotating truth frame to obtain a truth filtering result for each rotating anchor frame.

[0112] The truth value category can be the category of the objects contained in the rotated truth box. Different similarity calculation methods can be configured for different truth value categories. The similarity value between the rotated anchor box and the rotated truth box is used to detect whether the rotated anchor box is a positive truth sample for that rotated truth box. Generally, the higher the similarity value between the rotated anchor box and the rotated truth box, the more similar they are. The top k rotated anchor boxes with the highest similarity values ​​to the rotated truth box can be selected as positive truth samples for that rotated truth box, where k is greater than or equal to 1 and k is an integer. Similarly, the top g rotated anchor boxes with the lowest similarity values ​​to the rotated truth box can be selected as negative truth samples for that rotated truth box, where k is greater than or equal to 1 and k is an integer. The remaining rotated anchor boxes are ignored as truth samples for the rotated truth box. In addition, other judgment conditions can be used, which can be set according to the scenario and are not specifically limited.

[0113] By pre-setting different similarity calculation methods based on different truth categories, it is possible to adapt to different object detection scenarios. By using an adaptive approach to filter positive and negative samples, the accuracy of positive and negative sample filtering is improved, the representativeness of positive and negative samples is enhanced, and the object detection model is able to better learn the content of positive samples, thereby improving the object detection accuracy of the object detection model.

[0114] The truth value categories can be specifically divided into: categories with different lengths and widths, and categories with the same length and width.

[0115] Optionally, the similarity calculation method based on the truth categories of each of the rotated truth boxes, calculating the similarity value between each of the rotated anchor boxes and each of the rotated truth boxes, includes: for at least one of the rotated truth boxes i, when the truth category of the rotated truth box i includes different categories of length and width, calculating the rotation intersection-union ratio between the rotated truth box i and each of the rotated anchor boxes, as the similarity value between the rotated truth box i and each of the rotated anchor boxes, where i is a positive integer, i is less than or equal to N, and N is the number of the rotated truth boxes.

[0116] Anchor boxes, truth boxes, and the final prediction boxes are typically rectangles or rectangular shapes. Different aspect ratios can refer to object categories represented by rectangles with aspect ratios other than 1; for example, a rectangle with an aspect ratio other than 1 is a rectangular box.

[0117] For the i-th rotated truth box among N rotated truth boxes in the input data, if the rotated anchor box and rotated truth box i do not overlap, the rotation intersection-union ratio (ROU) is 0, and the similarity value is 0. If the rotated anchor box and rotated truth box i overlap, the ROU is calculated based on the overlapping area. Specifically, the overlapping area between the rotated anchor box and rotated truth box i is a polygonal region. This polygonal region can be divided into multiple triangular regions, and the area of ​​each triangular region is summed as the ROU between the rotated anchor box and rotated truth box i. N and i are both positive integers.

[0118] In scenarios where positive and negative samples are filtered using a positive / negative sample threshold, the presence of rotation angle deviations leads to low-precision similarity values ​​between the same rotated ground truth box i and multiple rotated anchor boxes when calculating similarity values. This results in similarity values ​​for these multiple anchor boxes being close and difficult to distinguish, making it hard to detect the rotating anchor box with the best match for the rotated ground truth box i as a positive sample. Furthermore, the greater the difference in length and width (the further the aspect ratio is from 1), the greater the rotation angle deviation, leading to lower accuracy in similarity values ​​calculated using cross-intersection over union (CUI) or center distance methods. By using Rotate Intersection over Union (ROU) to calculate similarity values ​​for rotated ground truth boxes of different length and width categories, the calculation accuracy of similarity values ​​is improved. This allows for precise differentiation of similarity values ​​between the same rotated ground truth box i and multiple rotated anchor boxes, thus identifying the rotating anchor box with the best match as a positive sample. This improves the anchor box prediction accuracy of the target detection model, thereby enhancing the overall target detection accuracy.

[0119] By calculating the rotation intersection-union ratio between the rotated truth box and the rotated anchor box, and using it as the similarity value between the two boxes, the similarity value can be accurately calculated. This avoids the problem of inaccurate positive and negative sample selection caused by large differences in matching thresholds due to different categories in length and width, thereby improving the accuracy of positive and negative sample selection.

[0120] Optionally, calculating the rotation intersection-union ratio (ROU) between the rotated truth frame i and each of the rotated anchor frames, as the similarity value between the rotated truth frame i and each of the rotated anchor frames, includes: calculating the distance between the rotated truth frame i and each of the rotated anchor frames for the rotated truth frame i; filtering out candidate anchor frames corresponding to the rotated truth frame i from among the rotated anchor frames based on the distance between the rotated truth frame i and each of the rotated anchor frames; and using the calculated ROU between the rotated truth frame i and each of the candidate anchor frames as the similarity value between the rotated truth frame i and each of the rotated anchor frames.

[0121] To filter rotated anchor frames, it's necessary to calculate the rotation intersection-union ratio (ROU) between each rotated anchor frame and the rotated truth frame i. However, this computation is computationally intensive, and there are numerous ignored and negative samples whose ROU calculations are redundant. To reduce redundant computation and improve the efficiency of similarity value calculation, we can first filter the rotated anchor frames. From the retained anchor frames, we can sequentially select those with the rotated truth frame i for similarity calculation. This significantly reduces the amount of data required for similarity calculation, and the complexity of the filtering calculation is lower than that of the similarity calculation, shortening the computation time for positive / negative similarity discrimination and improving the computational efficiency of positive / negative discrimination.

[0122] The distance between the rotated truth box i and the rotated anchor box can be the distance between the center point of the rotated truth box i and the center point of the rotated anchor box. Alternate anchor boxes can be those used in the rotation intersection-union ratio calculation with the rotated truth box i, or rotating anchor boxes that are similar to the rotated truth box i.

[0123] For each rotated truth box, the distances between rotated truth box i and each rotated anchor box are sorted. The top h rotated anchor boxes with the largest distances are selected as candidate anchor boxes for that rotated truth box i, where h is greater than or equal to 1 and is an integer. The rotation intersection-union ratio (ROI) is calculated between the rotated truth box i and each candidate anchor box to obtain the similarity value between the rotated truth box i and each candidate anchor box. Positive truth samples, negative truth samples, and ignored truth samples are then selected from the candidate anchor boxes. Furthermore, rotated anchor boxes other than the corresponding candidate anchor boxes can be considered as negative truth samples of the rotated truth box i. It should be noted that negative truth samples of the rotated truth box i can be positive truth samples of other rotated truth boxes (e.g., rotated truth box i+1).

[0124] By prioritizing the distance between each rotated truth frame and other rotated anchor frames, candidate anchor frames are selected. Then, the rotation intersection-union ratio (ROU) between the rotated truth frame and the candidate anchor frames is calculated as a similarity value to determine the positive or negative of the candidate anchor frames. In addition, the distance is also used to determine the positive or negative of rotated anchor frames other than the candidate anchor frames. This avoids calculating the ROU for each rotated anchor frame to obtain a similarity value for determining the rotated anchor frames, reducing the computational complexity of positive and negative determination, shortening the computation time of similarity positive and negative determination, and improving the computational efficiency of positive and negative determination.

[0125] Optionally, the similarity calculation method based on the truth categories of each of the rotated truth boxes, calculating the similarity value between each of the rotated anchor boxes and each of the rotated truth boxes, includes: for at least one of the rotated truth boxes p, when the truth category of the rotated truth box p includes the category of the same length and width, calculating the normalized distance between the center coordinates of the rotated truth box p and the center coordinates of each of the rotated anchor boxes, as the similarity value between the rotated truth box p and each of the rotated anchor boxes, where p is a positive integer, p is less than or equal to N, and N is the number of the rotated truth boxes.

[0126] The "same aspect ratio" category can refer to object categories represented by rectangles with an aspect ratio of almost 1, such as squares. The center coordinates can be the coordinates of a center point. For the p-th rotated truth box among the N rotated truth boxes in the input data, the normalized distance between the center coordinates of rotated truth box p and the center coordinates of each rotated anchor box is calculated as the similarity value between rotated truth box p and that rotated anchor box. Here, p is a positive integer.

[0127] For example, the normalized distance D between the center coordinates of a rotated truth box and the center coordinates of a rotated anchor box can be calculated using the following formula:

[0128]

[0129] Where Δx is the distance between the x-coordinate of the rotated truth box and the x-coordinate of the rotated anchor box; Δy is the distance between the y-coordinate of the rotated truth box and the y-coordinate of the rotated anchor box. Where 1.414 is... The approximate value of.

[0130] Furthermore, when the truth categories of each rotated truth box include categories with the same length and width, the cross-union ratio between each rotated truth box and each rotated anchor box can be calculated as a similarity value between each rotated truth box and each rotated anchor box.

[0131] For small-sized rotated ground truth boxes of the same category in both length and width, the intersection-over-union ratio (IoU) is insufficient to accurately distinguish between positive and negative samples. Normalized distance can quickly and accurately determine whether a rotated anchor box is a positive sample. For large-sized rotated ground truth boxes of the same category in both length and width, the IoU can be used as a similarity value.

[0132] By using the normalized distance between the rotated truth box and the rotated anchor box as the similarity value between them, the computational complexity of positive and negative discrimination can be reduced, while taking into account the accuracy of similarity value detection, thus improving the speed and accuracy of positive and negative discrimination.

[0133] Optionally, the truth value categories include categories with different lengths and widths and categories with the same length and width; the categories with different lengths and widths include motor vehicles and non-motor vehicles; the categories with the same lengths and widths include pedestrians and traffic cones.

[0134] The sample environment point cloud is a two-dimensional point cloud viewed from a bird's-eye view. Pedestrians and traffic cones can be identified by square boxes, corresponding to ground truth boxes of the same length and width. Motor vehicles and non-motor vehicles can be identified by rectangular boxes, corresponding to ground truth boxes of different lengths and widths.

[0135] By further subdividing categories with different lengths and widths into motor vehicles and non-motor vehicles, and further subdividing categories with the same length and width into pedestrians and traffic cones, object detection models can be trained for use in vehicle driving scenarios, improving the adaptability of object detection models and thus enhancing vehicle driving safety.

[0136] According to the technical solution disclosed herein, by rotating the ground truth box and the standard anchor box in a first manner, and training the model based on the rotated anchor box and the rotated ground truth box, it is possible to avoid re-laying and setting the anchor box and re-labeling the input data, simplifying the training operation of the model for object detection based on the rotated point cloud, accelerating the model training speed of the rotated point cloud, and improving the model training efficiency while increasing the detection range.

[0137] Figure 5 This is a flowchart of a target detection method disclosed in an embodiment of the present disclosure. This embodiment can be applied to detecting obstacles around a vehicle. The method of this embodiment can be executed by a target detection device, which can be implemented in software and / or hardware and specifically configured in an electronic device with a certain data processing capability. The electronic device can be a client device, such as a mobile phone, tablet computer, vehicle terminal, and desktop computer.

[0138] S501. Obtain the vehicle environment point cloud.

[0139] The vehicle collects point clouds of its surrounding environment and determines the collected vehicle environment point clouds. The vehicle can be an autonomous vehicle.

[0140] Optionally, acquiring the vehicle environment point cloud includes: acquiring the real-time point cloud collected by the vehicle; and converting the real-time point cloud into point cloud data from a bird's-eye view.

[0141] In a vehicle, radar installed on the vehicle collects point clouds of the surrounding environment in real time, resulting in a real-time point cloud. The vehicle then converts this real-time point cloud into a 2D point cloud from a bird's-eye view, obtaining the vehicle environment point cloud. From the driver's perspective while the vehicle is in motion, targets can occlude each other, affecting the accuracy of target detection. By converting the real-time point cloud from 3D space to a bird's-eye view, targets are separated from each other at a top-down angle, eliminating the occlusion problem and resulting in higher target detection accuracy.

[0142] By acquiring real-time point clouds collected by vehicles, the timeliness of the collected point cloud data is improved, enabling real-time target detection during autonomous driving. The real-time point cloud is converted into point cloud data from a bird's-eye view, avoiding the impact of mutual occlusion between targets on the target detection accuracy, further improving the accuracy of target detection. Furthermore, the conversion from 3D point cloud computing operations to 2D point cloud computing operations can reduce the amount of data involved in target detection and improve the target detection speed.

[0143] S502. Rotate the vehicle environment point cloud in a first manner to obtain a rotated vehicle environment point cloud, so that the point cloud in the vehicle environment point cloud at the position of the vehicle's preset movement direction is rotated to the position of the preset detection direction.

[0144] The preset movement direction is the vehicle's current driving direction, which is also directly in front of the vehicle. The target detection range of the object detection model is a square, and the preset detection direction is the detection direction corresponding to the diagonal of the square. The first rotation method can be a clockwise or counterclockwise rotation of 45 + n * 90, for example, 45 degrees.

[0145] S503. Extract the target size input data from the rotated vehicle environment point cloud and input it into the pre-trained target detection model to obtain the detection result output by the target detection model; the target size corresponds to the target detection range of the target detection model, the target detection range has different detection intervals in different directions, and the target detection range includes the detection interval corresponding to the preset detection direction.

[0146] The object detection model extracts features from the input data to obtain a first feature map. Based on the first feature map, a rotated anchor box is used to predict candidate boxes, their object categories, and the confidence scores of each candidate box. The predicted candidate boxes can be the result of a prediction obtained by rotating the anchor box. Candidate boxes with confidence scores greater than or equal to a preset confidence threshold are selected. Non-Maximum Suppression (NMS) is then used to filter these candidate boxes, resulting in target boxes and their object categories. Furthermore, in autonomous driving scenarios, different processing can be applied to different object categories. Features are extracted from the first feature map to obtain a second feature map. Based on the first feature map, a rotated anchor box is used to predict candidate boxes for small-sized categories, ultimately filtering out target boxes for small-sized categories and their object categories, such as pedestrians, bicycles, cones, or trash cans. Based on the second feature map, a rotated anchor box is used to predict candidate boxes for large-sized categories, ultimately filtering out target boxes for large-sized categories and their object categories, such as small or large vehicles.

[0147] S504. Rotate the target box in the detection result output by the target detection model in a second manner to obtain the target detection result. The second manner is the inverse of the first manner.

[0148] The bounding boxes output by the object detection model are the bounding boxes corresponding to the rotated vehicle environment point cloud. The bounding boxes can be rotated in reverse to obtain the bounding boxes corresponding to the unrotated vehicle environment point cloud. For example, the first method is a 45-degree clockwise rotation, and the second method is a 45-degree counter-clockwise rotation; alternatively, the first method is a 45-degree counter-clockwise rotation, and the second method is a 45-degree clockwise rotation. The rotation directions of the first and second methods are opposite, but the rotation angles are the same.

[0149] The target bounding box in the detection result is rotated, while the rest remains unchanged. The updated detection result is the target detection result.

[0150] Optionally, the target detection result may also include the category of the target bounding box.

[0151] The object detection model outputs detection results that also include the category of the bounding box. Based on the location of the bounding box, the relative position of the object included in the bounding box and the vehicle is determined; based on the category of the bounding box, the category of the object included in the bounding box is determined.

[0152] The categories of target bounding boxes can include other vehicles, pedestrians, animals, or roadside objects. The driving strategy required varies depending on the category of the target detection result. For example, other vehicles, pedestrians, or animals may be obstacles that appear in the lane or sidewalk, requiring the vehicle to avoid them; roadside objects, such as trash cans, mailboxes, or trees, are obstacles that do not appear in the lane or sidewalk, allowing the vehicle to drive normally along the lane without additional processing.

[0153] The driving strategy corresponding to the target detection result can be determined based on the category of the object included in the target bounding box. When it is necessary to avoid the target detection result, the vehicle can be controlled to avoid the object based on the position parameters and size parameters of the detection bounding box of the object in the point cloud data.

[0154] The target detection model trained above can quickly and accurately perceive obstacles around the vehicle and control the vehicle's movement accordingly, thereby improving driving safety.

[0155] According to the technical solution of this disclosure, by rotating the vehicle environment point cloud in a first manner to align the vehicle's movement direction with the detection direction of the detection interval corresponding to the preset detection direction, the detection interval can be adjusted according to the adaptability of the vehicle's movement direction, thereby flexibly adjusting the detection range in the movement direction to increase the detection range of the vehicle's movement direction, enabling low-cost perception of more distant obstacles, reducing the complexity of detecting distant obstacles, and quickly enabling the vehicle to detect more distant obstacles, thus improving vehicle driving safety. Furthermore, by using a second method, which is the inverse of the first method, to rotate the target box in the output result, the target box corresponding to the sample point cloud before rotation can be obtained, thereby reducing the computational complexity of the result after rotation due to rotation, reducing the complexity of the result caused by rotation, and achieving the goal of increasing the detection range while improving computational efficiency and improving the real-time performance of target detection.

[0156] In a specific application scenario, to ensure accurate ranging even for distant obstacles, this disclosure increases the detection range directly in front of the autonomous vehicle by rotating the entire point cloud of the autonomous driving scene. For example... Figure 2 As shown, after rotating the point cloud, the detection range of the autonomous vehicle is the same as the original. Rotating the point cloud will cause the point cloud distribution of the entire scene to change.

[0157] Before the scene rotated, most of the vehicles in the autonomous driving scene were parallel or perpendicular to each other, such as... Figure 2 In the left image, after rotating 45 degrees counterclockwise, the vehicles in the scene also rotate 45 degrees, as shown. Figure 2 The right image shows the significant difference in the orientation distribution of the ground truth bounding boxes before and after rotation, making it difficult for the object detection model to match high-quality positive samples during training. Furthermore, while approximate IoU calculations can reduce training time for relatively conventional bounding boxes before rotation, they offer lower matching accuracy for rotated objects, and using IoU with rotation incurs substantial computational overhead, impacting model training iterations. Therefore, without affecting the original obstacle detection accuracy or increasing training and inference latency, the model's view range is effectively extended.

[0158] 1. Re-cluster anchors based on rotated ground truth.

[0159] 2. Use rotated IoU instead of approximate IoU to select positive and negative samples for the model. In order to reduce invalid computation, the top-k anchors closest to the ground truth center point are selected first for sample matching. The matching threshold is refined when only necessary samples are calculated.

[0160] And, as Figure 4 As shown, the difference between the truth box and the corresponding anchor box of the obstacle before rotation is significantly smaller than the difference after rotation compared to the difference between the truth box and the corresponding anchor box of the obstacle after rotation. This applies to obstacles of varying lengths and widths, such as motor vehicles and non-motor vehicles. Figure 4 As shown, the difference increases before and after rotation. Furthermore, the intersection-union ratio (IUR) after rotation is a polygon, increasing computational complexity. This means obstacles with varying lengths and widths are more sensitive to rotation. Conversely, obstacles with the same length and width, such as pedestrians and cones, are less affected by rotation. Figure 4(Not shown in the image) is not sensitive to rotation, and the learning varies for different obstacle categories. Different matching methods are used for different types of obstacles: pedestrians and cones are selected using IoU or center-normalized distance for positive and negative sample selection; motor vehicles and non-motor vehicles are selected using rotated IoU for positive and negative sample selection.

[0161] Figure 2 Zuohe Figure 2 The right center represents the location of the autonomous vehicle. The innermost solid circle represents a 120m radius circle centered on the autonomous vehicle; the outermost solid circle represents a 160m radius circle. Since deep learning detection algorithms require regularly shaped square regions as input, assuming the autonomous vehicle's detection range is 120m in each direction (i.e., a 120x120 square frame), the autonomous driving perception algorithm can only detect obstacles within a maximum range of 120m. However, the diagonal of a 120x120 square frame is longer than its sides. Therefore, by rotating the point cloud of the entire scene, the detection area directly in front of the autonomous vehicle can be expanded from 120m to nearly [missing information] while keeping the input range unchanged. It can effectively improve the detection range directly in front.

[0162] This disclosure improves the detection range of the model by 41.67% (√2 times) without increasing latency, improves the recall of long-distance obstacles by more than 70%, and improves the target detection accuracy by more than 10%.

[0163] Figure 6 This is a flowchart of a vehicle control method disclosed according to an embodiment of the present disclosure. This embodiment can be applied to situations where vehicle control is performed based on target detection results of obstacles around the vehicle. The method of this embodiment can be executed by a vehicle control device, which can be implemented in software and / or hardware and specifically configured in an electronic device with a certain data processing capability. The electronic device can be a client device or a server device, such as a mobile phone, tablet computer, vehicle terminal, and desktop computer.

[0164] S601. Obtain the target detection result of the vehicle, wherein the target detection result is obtained by the target detection method as described in any embodiment of this disclosure.

[0165] S602. Based on the target detection result, control the vehicle to move.

[0166] Obtain the bounding boxes and their categories from the target detection results. Based on the bounding boxes and their categories, determine the corresponding driving strategy. When it is necessary to avoid objects included in the bounding boxes, the vehicle can be controlled to avoid the target detection results using the position and size parameters of the object's detection bounding box in the point cloud data, according to the avoidance method corresponding to the bounding box category.

[0167] By acquiring vehicle environment point clouds collected at multiple times, the vehicle can perceive the motion information of objects it encounters while driving. Based on the category and motion information of the perceived objects, the vehicle can adjust or generate navigation routes to avoid different types of obstacles using different driving modes, thereby improving driving safety.

[0168] According to the technical solution disclosed herein, by using the aforementioned target detection method and running the aforementioned trained target detection model, obstacles around the vehicle can be quickly and accurately perceived, and driving can be controlled accordingly to improve driving safety.

[0169] According to embodiments of this disclosure, Figure 7 This is a structural diagram of a training device for a target detection model according to an embodiment of this disclosure. This embodiment is applicable to training a target detection model for detecting obstacles around a vehicle. The device is implemented in software and / or hardware and is specifically configured in an electronic device with certain data processing capabilities.

[0170] like Figure 7 The training device 700 for an object detection model shown includes: a training sample input acquisition module 701, a point cloud rotation module 702, an input cropping module 703, and a model training module 704.

[0171] The training sample input acquisition module 701 is used to acquire training samples, which include a sample environment point cloud of the target device and at least one truth box.

[0172] The point cloud rotation module 702 is used to rotate the sample environment point cloud in a first manner to obtain the rotated sample environment point cloud, so that the point cloud in the sample environment point cloud at the position of the preset movement direction of the target device is rotated to the position of the preset detection direction.

[0173] The input cropping module 703 is used to crop the input data of the target size from the rotated sample environment point cloud, and input the input data into the target detection model to obtain the detection result output by the target detection model; the target size corresponds to the target detection range of the target detection model, the target detection range has different detection intervals in different directions, and the target detection range includes the detection interval corresponding to the preset detection direction;

[0174] The model training module 704 is used to adjust the parameters of the target detection model based on the difference between the detection results output by the target detection model and each of the ground truth boxes.

[0175] According to the technical solution of this disclosure, by rotating the sample environment point cloud in a first manner to align the detection direction of the detection interval corresponding to the preset movement direction with the preset detection direction, the corresponding detection interval can be adjusted for the adaptability of the movement direction, thereby flexibly adjusting the detection range in the movement direction to increase the detection range of the target device in the movement direction, realizing low-cost perception of more distant obstacles, reducing the complexity of detecting distant obstacles, and quickly enabling the target device to detect more distant obstacles, thereby improving the movement safety of the target device.

[0176] Furthermore, the detection interval corresponding to the preset detection direction includes: the longest detection interval.

[0177] Furthermore, the target detection range of the target detection model is a square, the preset movement direction before rotation is perpendicular to the edge of the square, and the preset detection direction is the detection direction corresponding to the diagonal of the square; the first method includes rotating the target device as the center along the first direction by a preset angle so that the preset movement direction after rotation is aligned with the detection direction corresponding to the diagonal of the square.

[0178] Furthermore, the input interception module 703 includes: a model input unit, used to input the input data into the target detection model; the target detection model, used to process the input data to obtain a feature map; the target detection model, used to generate multiple standard anchor boxes on the feature map and rotate them in the first manner to obtain multiple rotated anchor boxes; the target detection model, used to process the input data, determine the prediction filtering result of each of the rotated anchor boxes, and the regression prediction result and category prediction result of the rotated anchor boxes whose prediction filtering result is a positive sample, and use them as the detection result output by the target detection model; the model training module includes: the target detection model, used to adopt the... The first method rotates each of the ground truth boxes to obtain at least one rotated ground truth box; the target detection model is used to filter each of the rotated anchor boxes based on the difference between each of the rotated anchor boxes and each of the rotated ground truth boxes to obtain a ground truth filtering result for each of the rotated anchor boxes; the target detection model is used to calculate a first difference between the predicted filtering result of each of the rotated anchor boxes and the ground truth filtering result of each of the rotated anchor boxes; the target detection model is used to calculate a second difference between the regression prediction result and the category prediction result of the rotated anchor box whose predicted filtering result is a predicted positive sample and the corresponding rotated ground truth box; the target detection model is used to adjust the parameters of the target detection model based on the first difference and the second difference.

[0179] Furthermore, the target detection model includes: a ground truth category acquisition unit, used to acquire the ground truth category of each of the rotated ground truth boxes; a similarity value calculation unit, used to calculate the similarity value between each of the rotated anchor boxes and each of the rotated ground truth boxes based on the similarity calculation method corresponding to the ground truth category of each of the rotated ground truth boxes; and an anchor box filtering unit, used to filter each of the rotated anchor boxes according to the similarity value between each of the rotated anchor boxes and each of the rotated ground truth boxes, and obtain the filtering result corresponding to each of the rotated ground truth boxes.

[0180] Furthermore, the similarity calculation unit includes: a length-width difference similarity calculation subunit, used for calculating the rotation intersection-union ratio between the rotating truth box i and each of the rotating anchor boxes when the truth category of the rotating truth box i includes the length-width difference category, for at least one of the rotating truth boxes i. i is a positive integer, i is less than or equal to N, and N is the number of the rotating truth boxes.

[0181] Furthermore, the length and width different similarity calculation subunit includes: a distance detection subunit, used to calculate the distance between the rotated truth box i and each of the rotated anchor boxes for the rotated truth box i; an anchor box filtering subunit, used to filter out candidate anchor boxes corresponding to the rotated truth box i from each of the rotated anchor boxes based on the distance between the rotated truth box i and each of the rotated anchor boxes; and a similarity calculation subunit, used to calculate the rotation intersection-union ratio between the rotated truth box i and each of the candidate anchor boxes as the similarity value between the rotated truth box i and each of the rotated anchor boxes.

[0182] Furthermore, the similarity calculation unit includes: a similarity calculation subunit with the same length and width, used to calculate, for at least one of the rotating truth boxes p, when the truth category of the rotating truth box p includes the same length and width category, the normalized distance between the center coordinates of the rotating truth box p and the center coordinates of each of the rotating anchor boxes, as the similarity value between the rotating truth box p and each of the rotating anchor boxes, where p is a positive integer, p is less than or equal to N, and N is the number of the rotating truth boxes.

[0183] Furthermore, the truth value categories include categories with different lengths and widths and categories with the same length and width; the categories with different lengths and widths include motor vehicles and non-motor vehicles; the categories with the same lengths and widths include pedestrians and traffic cones.

[0184] The training apparatus for the above-mentioned target detection model can execute the training method of the target detection model provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the training method of the target detection model.

[0185] According to embodiments of this disclosure, Figure 8 This is a structural diagram of a target detection device according to an embodiment of this disclosure. This embodiment is applicable to detecting obstacles around a vehicle. The device is implemented using software and / or hardware and is specifically configured in an electronic device with certain data processing capabilities.

[0186] like Figure 8 The target detection device 800 shown includes: a point cloud data acquisition module 801, a point cloud data rotation module 802, an input data interception module 803, and a detection result rotation module 804.

[0187] in,

[0188] The point cloud data acquisition module 801 is used to acquire the vehicle environment point cloud.

[0189] The point cloud data rotation module 802 is used to rotate the vehicle environment point cloud in a first manner to obtain a rotated vehicle environment point cloud, so that the point cloud in the vehicle environment point cloud at the position of the vehicle's preset movement direction is rotated to the position of the preset detection direction.

[0190] The input data extraction module 803 is used to extract the target size input data from the rotated vehicle environment point cloud and input it into the pre-trained target detection model to obtain the detection result output by the target detection model; the target size corresponds to the target detection range of the target detection model, the target detection range has different detection intervals in different directions, and the target detection range includes the detection interval corresponding to the preset detection direction;

[0191] The detection result rotation module 804 is used to rotate the target box in the detection result output by the target detection model in a second manner to obtain the target detection result. The second manner is the inverse of the first manner.

[0192] According to the technical solution of this disclosure, by rotating the vehicle environment point cloud in a first manner to align the vehicle's movement direction with the detection direction of the detection interval corresponding to the preset detection direction, the corresponding detection interval can be adjusted for adaptability to the vehicle's movement direction, thereby flexibly adjusting the detection range in the movement direction to increase the detection range of the vehicle's movement direction, enabling low-cost perception of more distant obstacles, reducing the complexity of detecting distant obstacles, and quickly enabling the vehicle to detect more distant obstacles, thereby improving vehicle driving safety.

[0193] Furthermore, the point cloud data acquisition module 801 includes: a real-time point cloud acquisition unit for acquiring real-time point clouds collected by autonomous vehicles; and a top-down conversion unit for converting the real-time point clouds into point cloud data from a bird's-eye view.

[0194] Furthermore, the target detection device further includes: a category acquisition module for acquiring the category of the target box; and an autonomous driving module for controlling the autonomous vehicle to drive based on the target box and its category.

[0195] The target detection device described above can execute the target detection method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the target detection method.

[0196] Figure 9 This is a structural diagram of a vehicle control device disclosed in an embodiment of the present disclosure. This embodiment can be applied to situations where vehicle control is performed based on target detection results of obstacles around the vehicle. The device is implemented in software and / or hardware and is specifically configured in an electronic device with certain data processing capabilities.

[0197] like Figure 9 The vehicle control device 900 shown includes a target detection module 901 and a vehicle control module 902.

[0198] The target detection module 901 is used to acquire the target detection result of the vehicle, which is acquired by the target detection method as described in any embodiment of this disclosure.

[0199] The vehicle control module 902 is used to control the vehicle's movement based on the target detection results.

[0200] According to the technical solution disclosed herein, by using the aforementioned target detection method and running the aforementioned trained target detection model, obstacles around the vehicle can be quickly and accurately perceived, and driving can be controlled accordingly to improve driving safety.

[0201] The target detection device described above can execute the target detection method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the target detection method.

[0202] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0203] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0204] Figure 10 A schematic area diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0205] The electronic device provided in any embodiment of this disclosure can be applied to an autonomous driving system or to a platform that provides services for an autonomous driving system.

[0206] Optionally, the electronic device implementing the training method for the object detection model can be a training device for the object detection model. The electronic device implementing the object detection method can be an object detection device. The object detection device can be configured in an autonomous vehicle, specifically installed in the vehicle's onboard terminal. The electronic device implementing the vehicle control method can be a vehicle control device. The vehicle control device can be configured in an autonomous vehicle, specifically installed in the vehicle's onboard terminal.

[0207] Optionally, in addition to electronic devices, autonomous vehicles may also include communication components and environmental perception devices. The electronic devices can be integrated with the communication components or installed separately. The electronic devices can acquire data from environmental perception devices (such as cameras and radar), including images, videos, and point clouds, and then perform image, video, and point cloud processing and data calculations. Alternatively, the electronic devices themselves may also possess perception data acquisition and communication capabilities, such as an Artificial Intelligence (AI) camera. In this case, the electronic devices can directly perform image, video, and point cloud processing and data calculations based on the acquired perception data.

[0208] like Figure 10 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1002 or a computer program loaded from storage unit 1008 into random access memory (RAM) 1003. RAM 1003 may also store various programs and data required by device 1000 instructions. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0209] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0210] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as object detection model training methods, object detection methods, or vehicle control methods. For example, in some embodiments, the object detection model training methods, object detection methods, or vehicle control methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the object detection model training methods, object detection methods, or vehicle control methods described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured by any other suitable means (e.g., by means of firmware) to perform a training method for an object detection model, an object detection method, or a vehicle control method.

[0211] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard objects (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0212] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / instructions specified in the flowcharts and / or area diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0213] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0214] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0215] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0216] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem that addresses the management difficulties and weak business scalability inherent in traditional physical hosting and VPS services. Servers can also be servers for distributed systems or servers integrated with blockchain technology.

[0217] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0218] Cloud computing refers to a technology system that provides access to a shared pool of physical or virtual resources via a network. These resources can include servers, instruction sets, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for applications such as artificial intelligence and blockchain, as well as for model training.

[0219] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0220] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training an object detection model, comprising: Acquire training samples, which include a sample environment point cloud of the target device and at least one ground truth bounding box; The sample environment point cloud is rotated in a first manner to obtain a rotated sample environment point cloud, so that the point cloud in the sample environment point cloud at the position of the preset movement direction of the target device is rotated to the position of the preset detection direction. Input data of the target size is extracted from the rotated sample environment point cloud, and the input data is input into the target detection model to obtain the detection result output by the target detection model; The target size corresponds to the target detection range of the target detection model. The target detection range has different detection intervals in different directions. The target detection range includes the detection interval corresponding to the preset detection direction. The parameters of the target detection model are adjusted based on the difference between the detection results output by the target detection model and each of the ground truth boxes. The target detection model has a target detection range of a square, and the preset movement direction before rotation is perpendicular to the edge of the square. The preset detection direction is the detection direction corresponding to the diagonal of the square. The first method includes rotating the target device as the center along a first direction by a preset angle so that the preset movement direction after rotation is aligned with the detection direction corresponding to the diagonal of the square.

2. The method according to claim 1, wherein, The detection interval corresponding to the preset detection direction includes: the longest detection interval.

3. The method according to claim 1, wherein, The step of inputting the input data into the target detection model to obtain the detection result output by the target detection model includes: The input data is then fed into the target detection model; Using the target detection model, multiple standard anchor frames are generated on the input data and rotated in the first manner to obtain multiple rotated anchor frames; The target detection model processes the input data to determine the prediction and filtering results of each rotating anchor box, as well as the regression prediction results and category prediction results of the rotating anchor boxes that predict positive samples, and these are used as the detection results output by the target detection model. The step of adjusting the parameters of the target detection model based on the difference between the detection results output by the target detection model and each of the ground truth boxes includes: Using the target detection model, each of the truth boxes is rotated in the first manner to obtain at least one rotated truth box; Using the target detection model, each rotating anchor frame is filtered based on the difference between each rotating anchor frame and each rotating truth frame to obtain the truth filtering result of each rotating anchor frame; The first difference between the predicted filtering result of each of the rotating anchor frames and the true value filtering result of each of the rotating anchor frames is calculated using the target detection model. Using the target detection model, the regression prediction result and the category prediction result of the rotated anchor box that predicts positive samples are calculated, and the second difference between them and the corresponding rotated ground truth box is calculated. The parameters of the target detection model are adjusted based on the first difference and the second difference.

4. The method according to claim 3, wherein, The step of filtering each rotating anchor frame based on the difference between each rotating anchor frame and each rotating truth frame to obtain the truth filtering result of each rotating anchor frame includes: Obtain the truth category of each of the rotated truth boxes; Based on the similarity calculation method corresponding to the truth categories of each of the rotated truth boxes, the similarity value between each of the rotated anchor boxes and each of the rotated truth boxes is calculated; Based on the similarity value between each of the rotating anchor frames and each of the rotating truth frames, the rotating anchor frames are filtered to obtain the truth filtering results of each of the rotating anchor frames.

5. The method according to claim 4, wherein, The method of calculating the similarity between each rotated anchor box and each rotated truth box based on the similarity calculation method corresponding to the truth categories of each rotated truth box includes: For at least one of the rotated truth boxes i, when the truth category of the rotated truth box i includes different categories of length and width, the rotation intersection-union ratio between the rotated truth box i and each of the rotated anchor boxes is calculated as the similarity value between the rotated truth box i and each of the rotated anchor boxes, where i is a positive integer, i is less than or equal to N, and N is the number of the rotated truth boxes.

6. The method according to claim 5, wherein, The calculation of the rotation intersection-union ratio between the rotated truth frame i and each of the rotated anchor frames, as the similarity value between the rotated truth frame i and each of the rotated anchor frames, includes: For the rotated truth frame i, calculate the distance between the rotated truth frame i and each of the rotated anchor frames; Based on the distance between the rotated truth box i and each of the rotated anchor boxes, select the candidate anchor boxes corresponding to the rotated truth box i from each of the rotated anchor boxes; The similarity value between the rotated truth frame i and each of the candidate anchor frames is calculated based on the rotation intersection-union ratio.

7. The method according to claim 4, wherein, The method of calculating the similarity between each rotated anchor box and each rotated truth box based on the similarity calculation method corresponding to the truth categories of each rotated truth box includes: For at least one of the rotated truth boxes p, when the truth category of the rotated truth box p includes the category of the same length and width, the normalized distance between the center coordinates of the rotated truth box p and the center coordinates of each of the rotated anchor boxes is calculated as the similarity value between the rotated truth box p and each of the rotated anchor boxes, where p is a positive integer, p is less than or equal to N, and N is the number of the rotated truth boxes.

8. The method according to claim 4, wherein, The truth value categories include categories with different lengths and widths and categories with the same length and width; the categories with different lengths and widths include motor vehicles and non-motor vehicles; the categories with the same lengths and widths include pedestrians and traffic cones.

9. A target detection method, comprising: Obtain the vehicle's environmental point cloud; The vehicle environment point cloud is rotated in a first manner to obtain a rotated vehicle environment point cloud, so that the point cloud in the vehicle environment point cloud at the position of the vehicle's preset movement direction is rotated to the position of the preset detection direction. Input data of the target size is extracted from the rotated vehicle environment point cloud and input into a pre-trained target detection model to obtain the detection result output by the target detection model; The target size corresponds to the target detection range of the target detection model. The target detection range has different detection intervals in different directions. The target detection range includes the detection interval corresponding to the preset detection direction. The target detection range of the target detection model is a square. The preset movement direction before rotation is perpendicular to the edge of the square. The preset detection direction is the detection direction corresponding to the diagonal of the square. The first method includes rotating the target device as the center along a first direction by a preset angle so that the preset movement direction after rotation is aligned with the detection direction corresponding to the diagonal of the square. The target bounding box in the detection result output by the target detection model is rotated in a second way to obtain the target detection result. The second way is the inverse of the first way.

10. The method according to claim 9, wherein, The acquisition of the vehicle environment point cloud includes: Obtain the real-time point cloud data collected from the vehicle; The real-time point cloud is converted into point cloud data from a bird's-eye view.

11. A vehicle control method, comprising: The target detection result of the vehicle is obtained by the target detection method as described in any one of claims 9-10; Based on the target detection results, the vehicle is controlled to move.

12. A training device for an object detection model, comprising: The training sample input acquisition module is used to acquire training samples, which include a sample environment point cloud of the target device and at least one truth box; The point cloud rotation module is used to rotate the sample environment point cloud in a first manner to obtain the rotated sample environment point cloud, so that the point cloud in the sample environment point cloud at the position of the preset movement direction of the target device is rotated to the position of the preset detection direction. The input cropping module is used to crop the input data of the target size from the rotated sample environment point cloud, and input the input data into the target detection model to obtain the detection result output by the target detection model; The target size corresponds to the target detection range of the target detection model. The target detection range has different detection intervals in different directions. The target detection range includes the detection interval corresponding to the preset detection direction. The model training module is used to adjust the parameters of the target detection model based on the differences between the detection results output by the target detection model and each of the ground truth boxes; The target detection model has a target detection range of a square, and the preset movement direction before rotation is perpendicular to the edge of the square. The preset detection direction is the detection direction corresponding to the diagonal of the square. The first method includes rotating the target device as the center along a first direction by a preset angle so that the preset movement direction after rotation is aligned with the detection direction corresponding to the diagonal of the square.

13. The apparatus according to claim 12, wherein, The detection interval corresponding to the preset detection direction includes: the longest detection interval.

14. The apparatus according to claim 12, wherein, The input interception module includes: A model input unit is used to input the input data into the target detection model; The target detection model is used to process the input data to obtain a feature map; The target detection model is used to generate multiple standard anchor boxes on the feature map and rotate them in the first manner to obtain multiple rotated anchor boxes. The target detection model is used to process the input data, determine the prediction and filtering results of each of the rotating anchor boxes, and the regression prediction results and category prediction results of the rotating anchor boxes whose prediction and filtering results are positive samples, and use them as the detection results output by the target detection model. The model training module includes: The target detection model is used to rotate each of the truth boxes using the first method to obtain at least one rotated truth box; The target detection model is used to filter each rotating anchor frame based on the difference between each rotating anchor frame and each rotating truth frame, and obtain the truth filtering result of each rotating anchor frame. The target detection model is used to calculate the first difference between the predicted screening result of each of the rotating anchor frames and the true value screening result of each of the rotating anchor frames; The target detection model is used to calculate the second difference between the regression prediction result and the category prediction result of the rotated anchor box that predicts a positive sample and the corresponding rotated ground truth box. The target detection model is used to adjust the parameters of the target detection model based on the first difference and the second difference.

15. The apparatus according to claim 14, wherein, The target detection model includes: The truth category acquisition unit is used to acquire the truth category of each of the rotated truth boxes; The similarity calculation unit is used to calculate the similarity value between each of the rotating anchor frames and each of the rotating truth frames based on the similarity calculation method corresponding to the truth category of each of the rotating truth frames; An anchor frame filtering unit is used to filter each rotating anchor frame based on the similarity value between each rotating anchor frame and each rotating truth frame, and obtain the filtering result corresponding to each rotating truth frame.

16. The apparatus according to claim 15, wherein, The similarity calculation unit includes: The length and width different similarity calculation subunit is used to calculate the rotation intersection-union ratio between the rotating truth box i and each of the rotating anchor boxes when the truth value category of the rotating truth box i includes the length and width different categories. The ratio is used as the similarity value between the rotating truth box i and each of the rotating anchor boxes. i is a positive integer, i is less than or equal to N, and N is the number of the rotating truth boxes.

17. The apparatus according to claim 16, wherein, The similarity calculation subunit with different lengths and widths includes: The distance detection subunit is used to calculate the distance between the rotated truth box i and each of the rotated anchor boxes for the rotated truth box i. An anchor frame filtering subunit is used to filter out candidate anchor frames corresponding to the rotating truth frame i from each of the rotating anchor frames based on the distance between the rotating truth frame i and each of the rotating anchor frames. The similarity calculation subunit is used to calculate the rotation intersection-union ratio between the rotated truth box i and each of the candidate anchor boxes, and use it as the similarity value between the rotated truth box i and each of the rotated anchor boxes.

18. The apparatus according to claim 15, wherein, The similarity calculation unit includes: The similarity calculation subunit with the same length and width is used to calculate the normalized distance between the center coordinates of the rotating truth box p and the center coordinates of each of the rotating anchor boxes when the truth value category of the rotating truth box p includes the category of the same length and width. This normalized distance is used as the similarity value between the rotating truth box p and each of the rotating anchor boxes. p is a positive integer, p is less than or equal to N, and N is the number of rotating truth boxes.

19. The apparatus according to claim 15, wherein, The truth value categories include categories with different lengths and widths and categories with the same length and width; the categories with different lengths and widths include motor vehicles and non-motor vehicles; the categories with the same lengths and widths include pedestrians and traffic cones.

20. A target detection device, comprising: The point cloud data acquisition module is used to acquire the vehicle's environmental point cloud. The point cloud data rotation module is used to rotate the vehicle environment point cloud in a first manner to obtain the rotated vehicle environment point cloud, so that the point cloud in the vehicle environment point cloud at the position of the vehicle's preset movement direction is rotated to the position of the preset detection direction. The input data extraction module is used to extract the target size input data from the rotated vehicle environment point cloud and input it into the pre-trained target detection model to obtain the detection result output by the target detection model. The target size corresponds to the target detection range of the target detection model. The target detection range has different detection intervals in different directions. The target detection range includes the detection interval corresponding to the preset detection direction. The target detection range of the target detection model is a square. The preset movement direction before rotation is perpendicular to the edge of the square. The preset detection direction is the detection direction corresponding to the diagonal of the square. The first method includes rotating the target device as the center along a first direction by a preset angle so that the preset movement direction after rotation is aligned with the detection direction corresponding to the diagonal of the square. The detection result rotation module is used to rotate the target box in the detection result output by the target detection model in a second way to obtain the target detection result. The second way is the inverse of the first way.

21. The apparatus according to claim 20, wherein, The point cloud data acquisition module includes: A real-time point cloud acquisition unit is used to acquire the real-time point cloud data collected by the vehicle. The top-down conversion unit is used to convert the real-time point cloud into point cloud data from a bird's-eye view.

22. A vehicle control device, comprising: The target detection module is used to acquire the target detection result of the vehicle, wherein the target detection result is acquired by the target detection method as described in any one of claims 9-10; The vehicle control module is used to control the vehicle's movement based on the target detection results.

23. A training device for an object detection model, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the training method of the object detection model according to any one of claims 1-8.

24. A target detection device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the target detection method according to any one of claims 9-10.

25. A vehicle control device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the vehicle control method of claim 11.

26. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the training method of the target detection model according to any one of claims 1-8, the target detection method according to any one of claims 9-10, or the target detection method according to claim 11.

27. An autonomous vehicle, comprising the target detection device as claimed in claim 24 and / or the vehicle control device as claimed in claim 25.

28. A computer program product comprising a computer program that, when executed by a processor, implements a training method for an object detection model according to any one of claims 1-8, an object detection method according to any one of claims 9-10, or an object detection method according to claim 11.

Images