Target detection model training method, target detection method, device and vehicle

Abstract

The present invention relates to the field of image recognition technology, and specifically to a training method for a target detection model, a target detection method, a device, and a vehicle, with the purpose of solving the problem of type misdetection caused by incomplete feature information or noise interference. The training method of the target detection model includes: using the target detection model to determine the heat map corresponding to the sample image, based on the true category of the target to be identified in the heat map, determining key points from multiple position points included in the true box of the target to be identified, and optimizing the target detection model with the goal of maximizing the first loss, wherein each position point in the heat map shows the category probability of multiple object categories, and the first loss is used to characterize the difference between the first category probability corresponding to the true category and the second category probability not corresponding to the true category in the key point, and the true category is highlighted by increasing the difference between the first category probability and the second category probability, thereby improving the detection accuracy of the target detection model.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a target detection model training method, a target detection method, a device and a vehicle. Background Art

[0002] Object detection involves identifying specific objects in an image or video and determining their location and category. Three-dimensional (3D) object detection involves identifying and locating objects in three-dimensional space, outputting information such as the target's category, position, size, and orientation.

[0003] Related technologies address the issues of high noise and insufficient brightness in nighttime images by decomposing nighttime images and assigning weights based on the distance between pixel values ​​and the maximum pixel value. However, this approach, which assigns weights based on pixel values ​​in a feature map to highlight the target brightness range, can easily lose feature information, reducing target recognition accuracy due to incomplete information.

[0004] Another related technology constructs an infrared image segmentation function guided by the three objectives of noise removal, detail preservation, and edge preservation. It then uses a multi-objective optimization algorithm combined with a segmentation model to perform image segmentation and recognition within the reconstructed infrared image. This technical solution cannot accurately highlight targets and suppress noise in scenes with multiple objects and complex backgrounds. Furthermore, it relies on pixel segmentation to determine target category, which can easily lead to misclassification due to incomplete feature information or noise interference. Summary of the Invention

[0005] The present invention provides a target detection model training method, a target detection method, a device and a vehicle, which improve the detection accuracy of the target detection model.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present application provides a method for training a target detection model, the method comprising:

[0008] Use the target detection model to determine the heat map corresponding to the sample image.

[0009] Based on the true category of the target to be identified in the heat map, key points are determined from multiple position points included in the true box of the target to be identified.

[0010] Optimize the object detection model with the goal of maximizing the first loss.

[0011] Among them, each position point in the heat map shows the category probability of multiple object categories, and the first loss is used to characterize the difference between the first category probability corresponding to the true category in the key point and the second category probability that does not correspond to the true category.

[0012] The technical solution provided in the embodiments of the present application increases the target detection model's ability to distinguish between true categories and other categories by maximizing the difference between the category probabilities of the true category and other categories at the same spatial position on the heat map, enabling the target detection model to better determine the true category, significantly improving the classification accuracy of the target detection model, and is suitable for complex scenarios such as object occlusion and similar appearance.

[0013] One possible implementation involves determining key points from multiple locations within the ground-truth bounding box of the target, based on the target's true category in the heat map. Specifically, this can be accomplished by identifying the location with the highest probability of belonging to the first category within the ground-truth bounding box as the key point. This identification of the location with the highest probability of belonging to the first category within the ground-truth bounding box accurately describes the target's true location, maximizing the first loss at that location, and highlighting the target's true location and category, further improving the classification accuracy of the target detection model.

[0014] One possible implementation involves optimizing the target detection model with the goal of maximizing the first loss. This can be specifically achieved by optimizing the target detection model with the goal of maximizing both the first and second losses. The second loss characterizes the difference between the first-category probability of a keypoint and the first-category probabilities of its neighbors. By maximizing the difference between the first-category probabilities of a keypoint and its neighbors, the first-category probability of the keypoint is significantly higher than that of surrounding noise points, effectively suppressing noise interference, improving the robustness of the target detection model in complex environments, and reducing false detections caused by noise.

[0015] In one possible implementation, the calculation process of the first loss in the embodiment of the present application includes: determining the probability of a first category at a key point, determining the probability of a second category corresponding to at least one category other than the true category at the key point, determining at least one first difference between the probabilities of the first category and the at least one second category, and taking the sum of the absolute values ​​of the at least one first difference as the first loss. The calculation of the first loss is clear and efficient, enabling the object detection model to more accurately learn the relationships between objects, reducing overfitting and improving the model's generalization ability.

[0016] In one possible implementation, the calculation process of the second loss in the embodiment of the present application includes: determining the probability of the first category at the key point, determining the probability of the first category for at least one neighboring point of the key point, determining at least one second difference between the first category probability and the first category probability of the at least one neighboring point, and taking the sum of the absolute values ​​of the at least one second difference as the second loss. The calculation of the second loss is clear and efficient, enabling the object detection model to more accurately learn the relationship between objects, reducing overfitting and improving the model's generalization ability.

[0017] In one possible implementation, the target detection model includes a feature extraction layer, a feature fusion layer, and a target detection layer. Using the target detection model, the corresponding heatmap for a sample image is determined. This can be specifically implemented as follows: the feature extraction layer determines the multi-scale feature map of the sample image; the feature fusion layer aligns and fuses the multi-scale feature maps to obtain the target feature map; and the target detection layer determines the corresponding heatmap for the target feature map. By performing maximum pooling and alignment on the multi-scale feature maps, multi-scale information fusion is effectively achieved, thereby improving the detection accuracy and robustness of the target detection model.

[0018] One possible implementation involves using a feature fusion layer to align and fuse multi-scale feature maps to obtain a target feature map. Specifically, this can be achieved by downsampling the multi-scale feature maps through a pooling window in the feature fusion layer to align their sizes. These aligned multi-scale feature maps are then concatenated and fused to obtain the target feature map. Max pooling is then used to unify the sizes of the multi-scale feature maps. While aligning the multi-scale feature maps, the pooling process retains the most significant feature information, avoiding information loss and reducing false detections, further improving the accuracy of the target detection model.

[0019] In a second aspect, an embodiment of the present application provides a target detection method, the method comprising:

[0020] Get the image to be detected.

[0021] The image to be detected is input into the target detection model, and the target detection model predicts and outputs the target detection result corresponding to the image to be detected. The target detection model is trained based on the target detection model training method of the first aspect above.

[0022] The target detection method provided in the embodiment of the present application outputs target detection results using a target detection model trained by the target detection model training method of the first aspect mentioned above, thereby improving the detection accuracy of target detection.

[0023] In a third aspect, an embodiment of the present application provides a training device for a target detection model, the device including: a prediction module and an optimization module.

[0024] The above prediction module is used to determine the heat map corresponding to the sample image using the target detection model.

[0025] The above-mentioned optimization module is used to determine the key points from multiple position points included in the real box of the target to be identified based on the real category of the target to be identified in the heat map.

[0026] The above optimization module is also used to optimize the target detection model with the goal of maximizing the first loss.

[0027] Among them, each position point in the heat map shows the category probability of multiple object categories, and the first loss is used to characterize the difference between the first category probability corresponding to the true category in the key point and the second category probability that does not correspond to the true category.

[0028] In a possible implementation, the optimization module is configured to determine, as a key point, a position point having the highest probability of being in the first category among a plurality of position points included in the true bounding box.

[0029] In one possible implementation, the optimization module is configured to optimize the object detection model by maximizing a first loss and a second loss, wherein the second loss is used to represent the difference between the first category probability of the keypoint and the first category probabilities of the keypoint's neighbors.

[0030] In one possible implementation, the above-mentioned optimization module is used to: determine the probability of the first category at the key point, determine the probability of the second category corresponding to at least one category other than the true category at the key point, respectively determine at least one first difference between the first category and at least one second category probability, and take the sum of the absolute values ​​of the at least one first difference as the first loss.

[0031] In one possible implementation, the above-mentioned optimization module is used to: determine the first category probability at the key point, determine the first category probability of at least one neighborhood point of the key point, respectively determine at least one second difference between the first category probability and the first category probability of at least one neighborhood point, and take the sum of the absolute values ​​of the at least one second difference as the second loss.

[0032] In one possible implementation, the target detection model includes a feature extraction layer, a feature fusion layer, and a target detection layer. The prediction module is configured to: use the feature extraction layer to determine a multi-scale feature map of the sample image; use the feature fusion layer to align and fuse the multi-scale feature map to obtain a target feature map; and use the target detection layer to determine a heatmap corresponding to the target feature map.

[0033] In one possible implementation, the prediction module is configured to: utilize a feature fusion layer to downsample the multi-scale feature map through a pooling window to align the sizes of the multi-scale feature map; and concatenate and fuse the aligned multi-scale feature map to obtain a target feature map.

[0034] The technical effects corresponding to any one of the implementation methods in the third aspect can be referred to the technical effects corresponding to any one of the implementation methods in the above-mentioned first aspect, and will not be repeated here.

[0035] In a fourth aspect, an embodiment of the present application provides a target detection device, which includes an acquisition module and a detection module.

[0036] The acquisition module is used to acquire the image to be detected.

[0037] The detection module is configured to input an image to be detected into a target detection model, and output a target detection result corresponding to the image to be detected through prediction by the target detection model. The target detection model is trained based on the target detection model training method of the first aspect.

[0038] The technical effects corresponding to any one of the implementation methods in the fourth aspect can be referred to the technical effects corresponding to any one of the implementation methods in the above-mentioned second aspect, and will not be repeated here.

[0039] In a fifth aspect, an embodiment of the present application provides a computing device, which performs model training based on the target detection model training method of the first aspect above, or performs target detection based on the target detection method of the second aspect above.

[0040] In the sixth aspect, an embodiment of the present application provides a vehicle, which includes a vehicle body and the computing device of the fifth aspect above, or the training method of the target detection model of the first aspect above is applied to perform model training, or the target detection method of the second aspect above is applied to perform target detection.

[0041] In the seventh aspect, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the target detection model training method of the first aspect or the target detection method of the second aspect.

[0042] In an eighth aspect, a computer program product is provided, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, the training method of the target detection model of the first aspect or the target detection method of the second aspect is implemented.

[0043] The solutions provided in aspects 5 through 8 above implement the target detection model training method provided in aspect 1 above or the target detection method provided in aspect 2 above, and their specific implementations are not described in detail here. The technical effects corresponding to any implementation of the solutions provided in aspects 5 through 8 above can be found in the technical effects corresponding to any implementation of aspect 1 or aspect 2 above, and are not described in detail here.

[0044] It should be noted that various possible implementations of any of the above aspects can be combined under the premise that the solutions are not contradictory. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the structure of a computer system provided for an exemplary embodiment;

[0046] Figure 2 A flowchart of a method for training a target detection model provided by an exemplary embodiment;

[0047] Figure 3 A flowchart of a target detection method provided by an exemplary embodiment;

[0048] Figure 4 A schematic structural diagram of a training device for an object detection model provided by an exemplary embodiment;

[0049] Figure 5 A schematic structural diagram of a target detection device provided as an exemplary embodiment;

[0050] Figure 6 A schematic structural diagram of a computing device is provided for an exemplary embodiment. DETAILED DESCRIPTION

[0051] In order to enable ordinary people in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0052] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0053] In the embodiments of this application, words such as "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0054] First, the relevant technologies involved in this application are explained to facilitate understanding by those skilled in the art.

[0055] 3D object detection is a key research area in computer vision, widely used in fields such as autonomous driving, robotic navigation, augmented reality (AR), and medical imaging. Compared to traditional two-dimensional (2D) object detection, 3D object detection requires processing objects in three-dimensional space, involving complex features such as depth, position, and shape. Therefore, it faces greater technical challenges, particularly in object type identification and noise suppression. Existing 3D object detection technologies typically rely on depth information and image data acquired by laser detection and ranging (LiDAR), color-depth (RGB-Depth) cameras, or stereo cameras, combined with deep learning methods for object detection and localization. However, when objects have minimal differences in appearance, morphology, and spatial layout, existing object detection methods are prone to misidentifying one type of object as another. For example, in sparse or occluded scenes, the model may misclassify small objects as distant background or confuse objects of similar types. Furthermore, factors such as sensor noise, environmental complexity, and insufficient data collection often contribute to the generation of noise. Noise typically arises from sensor errors, object occlusion, or changes in viewing angle, leading to deviations in target recognition or missed detections. In scenes with multiple objects and complex backgrounds, noise is still difficult to completely suppress. Furthermore, the sizes of objects in 3D object detection vary significantly, and single-scale feature extraction methods cannot cover objects of all scales. Existing technologies extract multi-scale features through pyramid structures or scale-invariant convolutional neural networks (CNNs), but these methods are unable to effectively fuse them. Finally, object detection models can also experience false detections or missed detections when faced with unseen target types.

[0056] Based on this, the technical solution provided in the embodiment of the present application increases the target detection model's ability to distinguish between true categories and other categories by maximizing the difference between the category probabilities of true categories and other categories at the same spatial position on the heat map, so that the target detection model can better determine the true category, significantly improving the classification accuracy of the target detection model, and is suitable for complex scenarios such as object occlusion and similar appearance.

[0057] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0058] The solution provided in this application can be applied to Figure 1 In the computer system shown, Figure 1 As shown, the computer system provided by the embodiment of the present application includes a computing device 100. The computing device 100 can be a high-performance server, serving as the core of the computer system. The computing device 100 can deploy a target detection model, determine the heat map corresponding to the sample image 110 through the target detection model, and optimize the target detection model based on the heat map and the first loss. The computing device 100 can also optimize the target detection model based on the heat map, the first loss, and the second loss. The computing device 100 can also perform target detection by optimizing the trained target detection model and output a target detection result 120. The computing device 100 can also accept instructions from staff and flexibly configure the parameters involved in the target detection model based on the instructions.

[0059] Alternatively, the computing device 100 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, embedded hardware for real-time simulation, or a cloud server providing basic cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data. The embodiments of the present application do not limit the implementation and application scenarios of the computing device 100.

[0060] like Figure 2 As shown, the training method of the target detection model provided in the embodiment of the present application includes:

[0061] Step S201: The computing device uses the target detection model to determine the heat map corresponding to the sample image.

[0062] Heatmaps are used to display the object detection model's confidence in specific regions of a sample image (such as object locations, key points, or class activation regions), typically using color gradients to represent the response strength of different regions. In a heatmap, each location shows the class probabilities of multiple object categories, that is, the class probabilities of multiple channels for object detection. For example, sigmoid_heatmap[3,x,y]=0.9 indicates that the center point of the third class (such as a vehicle) exists at location (x,y), and the class probability corresponding to the third class is 0.9.

[0063] Exemplarily, the target detection model extracts and fuses multi-scale features of the sample image, and predicts an output heat map based on the fused multi-scale features.

[0064] Among them, multi-scale features are a set of features extracted from different scales (such as resolution or field of view) of the same sample image, which can capture diverse information in the sample image from local details to global structures.

[0065] In some embodiments, the object detection model includes a feature extraction layer, a feature fusion layer, and an object detection layer.

[0066] The feature extraction layer is used to extract multi-scale features from the sample image and determine the multi-scale feature map of the sample image. For example, the backbone network of the object detection model extracts multi-scale features of the sample image to obtain a multi-scale feature map.

[0067] Exemplarily, the feature extraction layer extracts multi-scale features using a convolutional neural network. Optionally, the convolutional neural network includes a residual network (ResNet), a deep layer aggregation network (DLA), an hourglass network, and a high-resolution network (HRNet).

[0068] The feature fusion layer is used to align and fuse multi-scale feature maps to obtain the target feature map.

[0069] Optionally, the feature fusion layer uses a pyramid network (FPN) to fuse multi-scale features from top to bottom, or a bidirectional feature pyramid network (BiFPN) to weightedly fuse features.

[0070] In some embodiments, the size alignment of the multi-scale feature maps is achieved through a maximum pooling operation, and the aligned multi-scale feature maps are spliced ​​along the channel dimension to generate a target feature map.

[0071] Exemplarily, the feature fusion layer is used to downsample the multi-scale feature maps through the pooling window to align the sizes of the multi-scale feature maps, retain the most significant feature information during the pooling process, and avoid information loss.

[0072] For example, the aligned multi-scale feature maps (such as n feature maps: ) for splicing, and fuse the spliced ​​multi-scale feature maps through the convolution layer to extract semantic information and generate the target feature map.

[0073] The target detection layer is used to determine the heat map corresponding to the target feature map.

[0074] Exemplarily, the target feature map is convolved through the target detection layer, for example: the number of channels is adjusted through 1 to 3 layers of lightweight convolution to generate a preliminary heat map response, the value range of the preliminary heat map response is compressed to [0, 1] to represent the category probability, and the heat map is determined.

[0075] In some embodiments, the target offset and width and height are generated by synchronous prediction through the target detection layer, and the category probability, target offset, width and height are combined to generate a prediction box.

[0076] Among them, the prediction box is a rectangular box predicted and generated by the target detection model, which is used to frame the target to be identified in the sample image and represent the category of the target to be identified.

[0077] Step S202: The computing device determines key points from a plurality of position points included in the real frame of the target to be identified based on the real category of the target to be identified in the heat map.

[0078] Among them, key points are used to accurately mark the parts of the target to be identified within the real frame, such as the eyes and nose in the face, the headlights and license plate center of the vehicle, etc.

[0079] Exemplarily, the position point with the highest probability of the first category among the multiple position points included in the true frame is determined as the key point.

[0080] Among them, the real box is a rectangular box that is manually annotated and accurately marks the target in the sample image.

[0081] The first category probability refers to the probability of the true category of the target being identified on the heatmap. For example, if the true category of the target being identified within the ground truth box is a vehicle, the probability of the vehicle category at each spatial location within the ground truth box is compared, and the spatial location with the highest probability of the vehicle category is identified as the keypoint.

[0082] Step S203: The computing device optimizes the target detection model with the goal of maximizing the first loss.

[0083] Among them, the first loss is used to characterize the difference between the probability of the first category corresponding to the true category in the key point and the probability of the second category that does not correspond to the true category.

[0084] The second-category probability refers to the probability of a keypoint belonging to a category other than the true category. For example, if the keypoints include vehicle, road, and road sign, and the true category is vehicle, the first loss represents the difference between the first-category probability for vehicle and the second-category probabilities for road and road sign.

[0085] Exemplarily, the probability of the first category at the key point is determined, and the probability of the second category corresponding to at least one category other than the true category at the key point is determined, and at least one first difference between the first category and at least one second category probability is determined respectively, and the sum of the absolute values ​​of the at least one first difference is taken as the first loss.

[0086] For example, within the prediction box (pred_box), the key points The true category The corresponding first category probability is pred_box , Represents the category (channel) index, then the first loss function can be expressed as the following formula:

[0087] .

[0088] in, represents the first loss, Represents the true category of the key point The corresponding first category probability, Indicates the key points except the true category Other The second category probability corresponding to the category.

[0089] In some embodiments, a negative value is taken for the first loss, and the target detection model is optimized with the goal of minimizing the first loss.

[0090] In some embodiments, the object detection model is optimized with the goal of maximizing the first loss and the second loss.

[0091] The second loss is used to characterize the difference between the first category probability of the key point and the first category probability of the neighborhood points of the key point.

[0092] Neighborhood points refer to other pixels or feature points within a local area (such as a circular or rectangular window) centered on the key point. For example, neighborhood points include four points on the diagonal of a rectangular area centered on the key point, as well as four points above, below, left, and right of the key point.

[0093] Optionally, the time distance between the domain point and the key point is a default value, or a manually set value, which can be flexibly adjusted according to specific needs.

[0094] Exemplarily, the first category probability at the key point is determined, as is the first category probability of at least one neighborhood point of the key point, and at least one second difference between the first category probability and the first category probability of at least one neighborhood point is determined respectively, and the sum of the absolute values ​​of the at least one second difference is taken as the second loss.

[0095] For example, within the prediction box (pred_box), the key points The first category probability is pred_box , key points With 8 neighborhood points, the second loss can be expressed as follows:

[0096] .

[0097] in, represents the second loss, represents the probability of the first category at the key point, Represents the probability of the first category of the j-th neighborhood point of the key point.

[0098] In some embodiments, a negative value is taken for the second loss, and the target detection model is optimized with the goal of minimizing the second loss.

[0099] In some embodiments, the prediction box parameters of the target detection model are adjusted to maximize the first loss and / or the second loss. Specifically, through the chain rule, the prediction box parameter gradient of the target detection model is transferred to the convolutional network of the target detection model, and the convolution kernel weights are updated to achieve optimization of the target detection model.

[0100] In some embodiments, during the optimization training process of the target detection model, the first loss, the second loss, the classification loss, and the regression loss are optimized simultaneously.

[0101] Exemplarily, through the back-propagation algorithm, the gradient accumulation of the first loss, the second loss, the classification loss and the regression loss is transmitted to each layer of the target detection model to guide the update of the target detection model parameters, so that the target detection model can accurately distinguish the target category to be identified, reduce type false detection, suppress noise, and improve the detection accuracy of the target detection model.

[0102] like Figure 3 As shown, the target detection method provided in the embodiment of the present application includes:

[0103] Step S301: The computing device obtains an image to be detected.

[0104] Step S302: The computing device inputs the image to be detected into the target detection model, and outputs the target detection result corresponding to the image to be detected through prediction by the target detection model.

[0105] Among them, the target detection model is trained based on the training method of the above target detection model.

[0106] Exemplarily, the target detection result is a heat map generated based on the prediction of the image to be detected, and the areas and categories of different categories of objects in the image to be detected are marked on the heat map.

[0107] In summary, the technical solution provided by the embodiments of this application effectively addresses the issues of false type detection and noise interference by introducing and maximizing the first and / or second losses during target detection model training, increasing the difference between the true type and other types at key points, as well as the difference between key points and neighboring points. Furthermore, when processing images, the target detection model aligns and fuses multi-scale features, addressing the multi-scale issue and significantly improving the accuracy, robustness, and generalization of the target detection model.

[0108] like Figure 4 As shown, the training device of the target detection model provided by the present application may include a prediction module 401 and an optimization module 402. The prediction module 401 is used to perform Figure 2 In the operation of step S201 of the illustrated method, the optimization module 402 is used to perform Figure 2 The operations of step S202 and step S203 in the illustrated method.

[0109] like Figure 5 As shown, the target detection device provided by the present application may include an acquisition module 501 and a detection module 502. The acquisition module 501 is used to execute Figure 3 In the operation of step S301 of the illustrated method, the detection module 502 is used to perform Figure 3 The operation of step S302 in the illustrated method.

[0110] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, the video quality assessment device or computing device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0111] The embodiment of the present application can, based on the above-mentioned video quality assessment method, exemplarily divide the video quality assessment device or computing device into functional modules. For example, the video quality assessment system or computing device can include various functional modules corresponding to the various functional divisions, or two or more functions can be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0112] like Figure 6 As shown, the computing device provided in the embodiment of the present application may include a processor 601, a bus 602, a communication interface 603, and a memory 604. The processor 601, the memory 604, and the communication interface 603 communicate with each other via the bus 602. It should be understood that the present application does not limit the number of processors and memories in the network device.

[0113] The bus 602 may be a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, or a universal serial bus (USB). Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 6 The bus 602 may include a path for transmitting information between various components of the network device (eg, memory 604, processor 601, communication interface 603).

[0114] The processor 601 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0115] The memory 604 may include volatile memory, such as random access memory (RAM). The processor 601 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0116] The communication interface 603 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the network device and other devices or a communication network.

[0117] The memory 604 stores executable program codes, and the processor 601 executes the executable program codes to implement the functions of the aforementioned method embodiments. That is, the memory 604 stores instructions for executing the aforementioned video quality assessment method.

[0118] The object detection model training method, object detection method, and computing device provided in the embodiments of the present application can be applied in vehicles. Vehicles can also be referred to as vehicles, mobile carriers, electric vehicles (EVs), hybrid electric vehicles (HEVs), plug-in hybrid electric vehicles (PHEVs), fuel cell vehicles (FCVs), autonomous vehicles, intelligent and connected vehicles (ICVs), driverless vehicles, etc.

[0119] In the embodiments of this application, the vehicle may be a sedan, a sport utility vehicle (SUV), a truck, an electric vehicle, a motorcycle, a tricycle, a special vehicle (such as an ambulance, fire truck, or police car), a driverless taxi, an intelligent connected bus, an autonomous logistics vehicle, an electric truck, etc. Furthermore, this method is also applicable to various specialized vehicles, such as agricultural vehicles, mining vehicles, forestry vehicles, airport vehicles, and port vehicles. This application does not impose any specific restrictions on this.

[0120] The target detection model training method, target detection method and computing device provided in the embodiments of the present application can also be applied to fields such as autonomous driving, robot navigation, and medical image processing, and have broad application prospects and high practical value.

[0121] An embodiment of the present application also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement a target detection model training method or a target detection method as provided in the above-mentioned method embodiments.

[0122] Alternatively, the computer-readable storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

[0123] An embodiment of the present application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, it implements the target detection model training method or target detection method provided in the above-mentioned method embodiments.

[0124] It should be noted that when the instructions in the above-mentioned computer-readable storage medium or one or more instructions in the computer program product are executed by the processor of the computing device, the various processes of the above-mentioned method embodiment are implemented and the same technical effect as the above-mentioned method can be achieved. To avoid repetition, they will not be repeated here.

[0125] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0127] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0128] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The software product is stored in a storage medium and includes a number of instructions for causing a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM, RAM, disk or optical disk, etc. Various media that can store program code.

[0130] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for training a target detection model, characterized in that: The method comprises: Determine a heat map corresponding to the sample image using the object detection model, wherein each location in the heat map shows the class probability of multiple object classes; Based on the true category of the target to be identified in the heat map, determining key points from a plurality of position points included in the true frame of the target to be identified; Optimizing the target detection model with the goal of maximizing the first loss and the second loss; Among them, the first loss is used to characterize the difference between the first category probability corresponding to the true category and the second category probability not corresponding to the true category in the key point; the second loss is used to characterize the difference between the first category probability of the key point and the first category probability of the neighborhood points of the key point.

2. The target detection model training method according to claim 1, characterized in that: The determining of key points from a plurality of position points included in a real frame of the target to be identified based on the real category of the target to be identified in the heat map includes: The position point with the highest probability of the first category among the multiple position points included in the real frame is determined as the key point.

3. The target detection model training method according to claim 1, characterized in that: The calculation process of the first loss includes: Determining a first category probability at the key point; Determining a second category probability corresponding to at least one category other than the true category at the key point; At least one first difference between the probabilities of the first category and at least one of the second categories is determined respectively, and the sum of the absolute values ​​of the at least one first difference is taken as the first loss.

4. The method for training a target detection model according to claim 1, wherein: The calculation process of the second loss includes: Determining a first category probability at the key point; Determine a first category probability of at least one neighboring point of the key point; At least one second difference between the first category probability and the first category probability of at least one neighborhood point is determined respectively, and a sum of absolute values ​​of the at least one second difference is taken as the second loss.

5. The method for training a target detection model according to claim 1, wherein: The target detection model includes a feature extraction layer, a feature fusion layer and a target detection layer; Determining the heat map corresponding to the sample image using the target detection model includes: Determine a multi-scale feature map of the sample image using the feature extraction layer; Using the feature fusion layer, feature alignment and fusion are performed on the multi-scale feature maps to obtain a target feature map; The target detection layer is used to determine a heat map corresponding to the target feature map.

6. The method for training a target detection model according to claim 5, wherein: The step of utilizing the feature fusion layer to perform feature alignment and fusion on the multi-scale feature map to obtain a target feature map includes: Downsampling the multi-scale feature map through a pooling window using the feature fusion layer to align the sizes of the multi-scale feature map; The aligned multi-scale feature maps are spliced ​​and fused to obtain the target feature map.

7. A target detection method, characterized in that: The target detection method comprises: Obtain the image to be detected; The image to be detected is input into a target detection model, and the target detection model predicts and outputs a target detection result corresponding to the image to be detected; wherein, the target detection model is trained based on the training method of the target detection model described in any one of claims 1 to 6.

8. A training device for a target detection model, characterized in that: The device comprises: A prediction module, configured to use an object detection model to determine a heat map corresponding to a sample image, wherein each location in the heat map shows a class probability of multiple object classes; An optimization module, configured to determine key points from a plurality of position points included in a real frame of the target to be identified based on a real category of the target to be identified in the heat map; The optimization module is further configured to optimize the target detection model with the goal of maximizing the first loss and the second loss; Among them, the first loss is used to characterize the difference between the first category probability corresponding to the true category and the second category probability not corresponding to the true category in the key point; the second loss is used to characterize the difference between the first category probability of the key point and the first category probability of the neighborhood points of the key point.

9. A target detection device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be detected; A detection module is used to input the image to be detected into a target detection model, and predict and output the target detection result corresponding to the image to be detected through the target detection model; wherein, the target detection model is trained based on the training method of the target detection model described in any one of claims 1 to 6.

10. A computing device, characterized in that The computing device applies the target detection model training method according to any one of claims 1 to 6 to train the target detection model, or applies the target detection method according to claim 7 to perform target detection.

11. A vehicle, characterized in that: The vehicle includes a vehicle body and a computing device as described in claim 10; or, the target detection model is trained using the target detection model training method as described in any one of claims 1 to 6; or, the target detection method as described in claim 7 is applied to perform target detection.

Images