Training method, device, equipment and storage medium for target detection model

Abstract

The present application discloses a training method, device, equipment and storage medium for a target detection model. It can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving. The method includes: obtaining at least one sample image and the annotation information corresponding to the sample image; performing data enhancement processing on the sample image to obtain at least one processed image; obtaining first information and second information according to the sample image through the target detection model; obtaining third information and fourth information corresponding to each processed image according to the first model according to at least one processed image; obtaining pseudo-annotation information corresponding to the annotation information according to the third information and fourth information corresponding to each processed image; adjusting the parameters of the target detection model according to the first information, the second information, the annotation information and the pseudo-annotation information to obtain a trained target detection model. The above method can weaken the influence of the annotation data noise on the model performance.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a training method, apparatus, device and storage medium for a target detection model. Background Art

[0002] Object detection refers to the process of automatically identifying and locating specific objects in images or videos. In related technologies, an object detection model trained by a deep learning algorithm can be used to complete the above process.

[0003] However, during the training process of the object detection model, the performance of the object detection model may be negatively affected due to the presence of annotation data noise such as label noise (i.e., the category label of the object is incorrectly marked as other categories) and border noise (i.e., the position of the annotation box indicating the location of the object is inaccurate) in the training data.

[0004] In the related art, usually only one of the above-mentioned label noise and border noise can be suppressed, and the influence of the other noise on the model training process cannot be avoided, resulting in limited performance of the target detection model. Summary of the invention

[0005] The embodiment of the present application provides a training method, apparatus, device and storage medium for a target detection model. The technical solution provided by the embodiment of the present application is as follows:

[0006] According to one aspect of an embodiment of the present application, a method for training a target detection model is provided, the method comprising:

[0007] Acquire at least one sample image and annotation information corresponding to the sample image; wherein the sample image includes at least one detection target, and the annotation information includes label information and position information of the detection target, the label information is used to indicate the category of the detection target, and the position information is used to indicate the position of the detection target in the sample image;

[0008] Performing data enhancement processing on the sample image to obtain at least one processed image;

[0009] Obtaining first information and second information according to the sample image through a target detection model; wherein the first information is used to indicate a prediction result of the target detection model on the category of the detection target, and the second information is used to indicate a prediction result of the target detection model on the position of the detection target in the sample image;

[0010] Obtaining third information and fourth information corresponding to each of the processed images respectively according to the at least one processed image by the first model; wherein the third information is used to indicate the prediction result of the first model for the category of the detection target, and the fourth information is used to indicate the prediction result of the first model for the position of the detection target in the processed image;

[0011] According to the third information and the fourth information respectively corresponding to each of the processed images, pseudo-annotation information corresponding to the annotation information is obtained, wherein the pseudo-annotation information includes pseudo-label information and pseudo-position information, the pseudo-label information is used to indicate the category of the detection target, and the pseudo-position information is used to indicate the position of the detection target in the sample image;

[0012] According to the first information, the second information, the labeling information and the pseudo-labeling information, the parameters of the target detection model are adjusted to obtain a trained target detection model.

[0013] According to one aspect of an embodiment of the present application, a training device for a target detection model is provided, the device comprising:

[0014] An acquisition module, configured to acquire at least one sample image and annotation information corresponding to the sample image; wherein the sample image includes at least one detection target, and the annotation information includes label information and position information of the detection target, wherein the label information is used to indicate the category of the detection target, and the position information is used to indicate the position of the detection target in the sample image;

[0015] An enhancement module, used for performing data enhancement processing on the sample image to obtain at least one processed image;

[0016] A target detection module, configured to obtain first information and second information according to the sample image through a target detection model; wherein the first information is used to indicate a prediction result of the target detection model on the category of the detection target, and the second information is used to indicate a prediction result of the target detection model on the position of the detection target in the sample image;

[0017] A first detection module, configured to obtain third information and fourth information respectively corresponding to each of the processed images according to the at least one processed image through a first model; wherein the third information is used to indicate a prediction result of the first model for the category of the detection target, and the fourth information is used to indicate a prediction result of the first model for the position of the detection target in the processed image;

[0018] a pseudo-annotation generating module, configured to obtain pseudo-annotation information corresponding to the annotation information according to the third information and the fourth information respectively corresponding to each of the processed images, wherein the pseudo-annotation information includes pseudo-label information and pseudo-position information, wherein the pseudo-label information is used to indicate the category of the detection target, and the pseudo-position information is used to indicate the position of the detection target in the sample image;

[0019] An adjustment module is used to adjust the parameters of the target detection model according to the first information, the second information, the labeling information and the pseudo-labeling information to obtain a trained target detection model.

[0020] According to one aspect of an embodiment of the present application, a computer device is provided, which includes a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the above-mentioned target detection model training method.

[0021] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned target detection model training method.

[0022] According to one aspect of an embodiment of the present application, a computer program product is provided, which includes a computer program stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the above-mentioned target detection model training method.

[0023] The technical solution provided by the embodiments of the present application includes at least the following beneficial effects:

[0024] By obtaining a sample image including at least one detection target and annotation information that can indicate the category and position of the detection target in the sample image. And, using the target detection model and the first model respectively, according to the sample image and the processed image obtained by data enhancement processing of the sample image, predict the category and position of the detection target. Afterwards, according to the prediction result of the first model, pseudo-annotation information that can also be used to indicate the category and position of the detection target in the sample image is obtained. Finally, according to the above-mentioned annotation information, pseudo-annotation information and the prediction result of the target detection model, the parameters of the target detection model are adjusted to obtain the trained target detection model. By training the target detection model by the above method, the influence of the noise of the category error and the position error of the detection target in the annotation information on the model training process can be effectively weakened, and there is no need to screen the annotation information, and all training resources can be fully utilized, thereby improving the model performance. At the same time, this method has no specific requirements for the structure of the target detection model and has good generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

[0026] Figure 2 is a flow chart of a method for training a target detection model provided by an embodiment of the present application;

[0027] Figure 3 It is a schematic diagram of a complete solution of a method for training a target detection model provided by an embodiment of the present application;

[0028] Figure 4 It is a schematic diagram of test results of using different training methods for the same model provided by an embodiment of the present application;

[0029] Figure 5 is a block diagram of a training device for a target detection model provided by an embodiment of the present application;

[0030] Figure 6 It is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0032] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0033] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0034] Computer vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further perform image processing so that the computer processes the images into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Large model technology has brought important changes to the development of computer vision technology. Pre-trained models in the field of vision such as Swin-Transformer, ViT (Vision Transformer, visual self-attention model), V-MOE (Vision Mixture of Experts, mixed expert visual model), MAE (Masked Autoencoder, masked autoencoder) can be quickly and widely applied to downstream specific tasks after fine tuning. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D (three-dimensional) technology, virtual reality, augmented reality, simultaneous positioning and map construction, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0035] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning. The pre-trained model is the latest development in deep learning, which integrates the above technologies.

[0036] The technical solutions provided in the embodiments of the present application involve technologies such as artificial intelligence computer vision and machine learning, which are specifically introduced and explained through the following embodiments.

[0037] In the field of computer vision, object detection has a wide range of applications, such as autonomous driving, smart home, etc. Object detection usually includes two main steps: object localization and object classification. Object localization refers to locating the position of the object in the image, usually using a bounding box or mask to represent the location of the object. Object classification refers to matching the located object with a predefined object category to determine the category of the object.

[0038] In the related art, annotated data can be used to train deep learning models so that the deep learning models have the ability to detect targets. Annotation data noise refers to the erroneous or inaccurate annotation information in the annotation data due to subjective factors of the annotators, limitations of the annotation tools, or other factors. Annotation data noise generally includes two main categories: label noise and border noise. Label noise refers to the incorrect labeling of an object as another category, while border noise refers to the inaccurate position of the annotation frame, which is not close to the edge of the object. These noises may cause negative effects on the training and performance of deep learning models, so corresponding methods need to be taken to combat these noises to improve the accuracy and robustness of the model.

[0039] In response to the impact of annotation noise in training data on the performance of deep learning models, related technologies usually deal with label noise and border noise separately. Reducing the impact of either type of noise can improve the performance of the model.

[0040] For the annotated label noise, there are three main processing solutions in related technologies: one is to design a training loss function with noise resistance; the second is to design a model structure that is more suitable for processing noise; the third is to design a sampling method to remove noisy training data.

[0041] For the annotated border noise, the relevant technology mainly reduces the impact of noise by adding branches to the detection head of the model, estimating the accuracy of the border obtained by model reasoning, and adjusting the border regression loss function to reduce the weight of the noise annotation.

[0042] However, the above solutions all have their own shortcomings:

[0043] 1. For label noise, the beneficial effects of designing and changing the loss function are usually limited, and the change of the loss function also changes the optimization goal of the model. Although it reduces the impact of noisy label data, it is likely to affect the improvement of the model's detection ability.

[0044] 2. Although the method of designing a new anti-noise model structure for label noise is effective, it requires changing the model structure, which loses the versatility of the method and cannot be directly applied to existing fixed models and methods. The flexibility and convenience of application are limited, and adding model components may increase the inference hardware resource overhead when the model is applied.

[0045] 3. Regarding label noise, we remove noisy labeled data and select clean training data. Although this method can reduce the impact of label noise on the model, the removed labeled data also contains rich information, and its bounding box annotations are still correct. Although direct discarding eliminates certain noise information, it also discards some meaningful information. The performance of noisy model training is limited, and the cost of labeling resources required to achieve the same model performance increases.

[0046] 4. Regarding border noise, the impact of label noise is not comprehensively considered, and only border noise can be processed alone, so the effectiveness of the solution is not high.

[0047] The technical solution provided in the embodiment of the present application can effectively process both label noise and border noise without increasing the model structure and retaining all training data. This will be described in detail in the following embodiment.

[0048] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment may include a model training device 10 and a model using device 20.

[0049] The model training device 10 may be an electronic device such as a mobile phone, a computer, a vehicle terminal, an aircraft, a server, an intelligent robot, a smart TV, an intelligent voice interaction device, a multimedia playback device, or other electronic devices with strong computing capabilities, which is not limited in this application. The model training device 10 is used to train the target detection model 30 and adjust the first model 40.

[0050] In the embodiment of the present application, the target detection model 30 is a machine learning model. Optionally, the model training device 10 can train the target detection model 30 in a machine learning manner so that it has better performance.

[0051] Optionally, the training process of the target detection model 30 is as follows: a sample image including at least one detection target and annotation information that can indicate the category and position of the detection target in the sample image are obtained. Through the target detection model 30, first information and second information are obtained according to the sample image, and the first information and the second information are respectively used to indicate the prediction results of the target detection model 30 on the category of the detection target and its position in the sample image. Through the first model 40, according to at least one processed image obtained by performing data enhancement processing on the sample image, third information and fourth information corresponding to each processed image are obtained, and the third information and the fourth information are respectively used to indicate the prediction results of the first model 40 on the category of the detection target and its position in the processed image. According to the third information and the fourth information corresponding to each processed image, pseudo-annotation information that can also be used to indicate the category and position of the detection target in the sample image is obtained. According to the pseudo-annotation information, the annotation information, the first information and the second information, the parameters of the target detection model 30 are adjusted to obtain a trained target detection model. Optionally, the parameters of the first model 40 are adjusted according to the parameters of the trained target detection model to obtain an adjusted first model, and the adjusted first model is used to participate in the next round of training of the trained target detection model.

[0052] In some embodiments, the model using device 20 can be an electronic device such as a mobile phone, a computer, a vehicle terminal, an aircraft, a server, an intelligent robot, a smart TV, an intelligent voice interaction device, a multimedia playback device, or other electronic devices with strong computing power, which is not limited in this application. The trained target detection model can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0053] Below, some examples are used to introduce the specific application of the trained target detection model:

[0054] 1. Autonomous driving: used to identify vehicles, pedestrians, traffic signs, etc. on the road.

[0055] 2. Smart home: used to identify family members, pets, objects, etc.

[0056] 3. Medical imaging: used for medical image analysis, such as tumor detection, lesion identification, etc.

[0057] 4. Industrial manufacturing: used to detect and identify products and parts on the production line.

[0058] 5. Game development: used to realize target detection and search of virtual characters, etc.

[0059] The model training device 10 and the model using device 20 can be two independent devices or the same device.

[0060] In the method provided in the embodiment of the present application, the execution subject of each step may be a computer device, which refers to an electronic device with data calculation, processing and storage capabilities. Wherein, when the computer device is a server, the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The computer device may be Figure 1 The model training device 10 may also be a model using device 20.

[0061] Please refer to Figure 2 , which shows a flow chart of a method for training a target detection model provided by an embodiment of the present application. The execution subject of each step of the method may be the model training device 10 introduced above. In the following method embodiment, for the sake of ease of description, only the execution subject of each step is introduced as a "computer device". The method may include at least one of the following steps 210 to 260.

[0062] Step 210: Obtain at least one sample image and annotation information corresponding to the sample image.

[0063] The sample image includes at least one detection target, and the annotation information includes label information and position information of the detection target, the label information is used to indicate the category of the detection target, and the position information is used to indicate the position of the detection target in the sample image.

[0064] The sample image can be a picture or an image frame in a video, which is not limited in this application. The detection target refers to the object that needs to be located and classified in the sample image. For example, in the field of unmanned driving, the detection target may include people, animals, or some other obstacles on the road. In different application scenarios, technicians can select different detection targets, which is not limited in this application.

[0065] The annotation information corresponds to the sample image and is used to indicate the category and position of the detection target in the sample image. In some embodiments, the label information can be directly indicated by text such as "person" or "car", or the label information can be in the form of a vector, each dimension of the vector corresponds to a different object category, and when the value of a dimension in the vector is 1 and the values ​​of the remaining dimensions are 0, it indicates that the category of the detection target is the category corresponding to the dimension. The position information can be any information that can indicate the position of the detection target in the sample image, for example, the coordinate values ​​of the four vertices of the rectangular box that surrounds the detection target, or some other coordinates or vector values ​​that can indicate the position of the detection target.

[0066] Step 220: Perform data enhancement processing on the sample image to obtain at least one processed image.

[0067] In some embodiments, data enhancement processing is performed on sample images, which can also be described as perturbation processing of sample images. Data enhancement refers to obtaining more images containing effective information based on the original sample images through image processing, thereby enriching the distribution of training data.

[0068] In some embodiments, the data enhancement processing includes at least one of the following: transforming the size of the sample image, flipping the sample image, and changing the luminosity of the sample image. That is, the processed image can be generated by randomly transforming the size of the sample image, randomly flipping the image, and randomly perturbing the luminosity. Exemplarily, the ratio of the randomly transformed image size can range from 0.5 to 1.5 in both the horizontal and vertical directions, the probability of randomly flipping the image can be set to 0.5, the brightness change amplitude of the random luminosity perturbation can be 10, the contrast value range can be 0.9 to 1.1, and the saturation value range can be 0.9 to 1.1.

[0069] In some embodiments, the data enhancement process further includes at least one of the following: rotating the sample image, cropping the sample image, adding a mask or Gaussian noise to the sample image, which is not limited in the present application.

[0070] The above method performs data enhancement processing on the sample image to obtain at least one processed image for pseudo label generation. Since each processed image is generated by a different method, they all have valid and different image information. When the first model performs target detection on these processed images respectively, different prediction results can be obtained. Since the pseudo label information is generated based on these different prediction results, pseudo label information with less data noise (i.e., it can more accurately indicate the category and position of the detection target) can be finally obtained.

[0071] In addition, it should be noted that, since each processed image has valid and different image information, the more processed images there are, the better the noise reduction effect that can be achieved by this solution (i.e., the less data noise there is in the pseudo-label information). However, in related experiments, when the number of processed images is greater than 8, the marginal gain of noise reduction is not obvious. Therefore, technicians can freely set the number of processed images that can be obtained from a single sample image according to their needs, and this application does not limit this.

[0072] Step 230, obtaining first information and second information according to the sample image through the target detection model.

[0073] The first information is used to indicate the prediction result of the target detection model on the category of the detection target, and the second information is used to indicate the prediction result of the target detection model on the position of the detection target in the sample image.

[0074] In an embodiment of the present application, the target detection model can be any model in the field of target detection, for example, RCNN (Region Convolutional Neural Network) series models, YOLO (You Only Look Once) series models, etc.

[0075] In some embodiments, the first information may include a third vector, which is used to characterize the prediction result of the target detection model on the probability distribution of the category of the detection target. For example, if the target detection model divides the category of the detection target into three categories, namely "human", "animal", and "other objects", and the third vector is (0.1, 0.8, 0.1), it can be indicated that the model predicts that the probability of the detection target being a "human" is 0.1, the probability of being an "animal" is 0.8, and the probability of being an "other object" is 0.1. Furthermore, the first information may also include a classification score, which may also be expressed in vector form. The higher the value of a dimension in the vector (that is, the higher the score), the higher the probability that the target detection model predicts that the category of the detection target is the category corresponding to the dimension.

[0076] In some embodiments, the second information includes coordinates or vector values ​​that can indicate the predicted position of the detection target in the sample image, for example, coordinate values ​​of a mask that covers the detection target, and for example, coordinate values ​​of four vertices of a rectangular box that surrounds the detection target. This application is not limited to this.

[0077] In the process of the target detection model predicting the coordinates, a probability distribution is actually predicted for the coordinates first, and then the coordinate values ​​included in the second information are determined based on the probability distribution. For example, the mean of the predicted Gaussian distribution is determined as the coordinate value included in the second information. Therefore, in some embodiments, the second information also includes the coordinate variance corresponding to the above coordinates (i.e., the variance corresponding to the above probability distribution), and the coordinate variance here is used to indicate the degree of discreteness of the probability distribution of the above coordinates predicted by the target detection model.

[0078] Step 240: Obtain third information and fourth information corresponding to each processed image according to at least one processed image through the first model.

[0079] The third information is used to indicate the prediction result of the first model for the category of the detection target, and the fourth information is used to indicate the prediction result of the first model for the position of the detection target in the processed image.

[0080] In the embodiment of the present application, the structure of the first model is the same as that of the target detection model, and therefore, the first model may also be any model in the field of target detection.

[0081] In addition, it should be noted that in the embodiment of the present application, since the processed image is obtained based on the sample image, the information indicating the position of the detection target in the processed image can also indicate the position of the detection target in the sample image.

[0082] In some embodiments, the third information may include a first vector, where the first vector is used to represent a prediction result of the first model on the probability distribution of the category of the detection target.

[0083] In some embodiments, the fourth information includes coordinates or vector values ​​that can indicate the predicted position of the detection target in the processed image, for example, the coordinate values ​​of a mask that masks the detection target. Further, the fourth information may include detection frame coordinates, and the detection frame is used to indicate the position of the detection target in the processed image. The detection frame can be a rectangular frame with four vertices in the above embodiment, or a frame of other shapes that can surround the detection target, and this application does not limit this.

[0084] In addition, in some embodiments, the fourth information also includes the coordinate variance corresponding to the above-mentioned detection frame coordinates, and the coordinate variance corresponding to the detection frame coordinates is used to indicate the degree of discreteness of the probability distribution of the detection frame coordinates predicted by the first model. In other words, the coordinate variance corresponding to the detection frame coordinates refers to the variance corresponding to the probability distribution of the detection frame coordinates predicted by the first model. Exemplarily, if the probability distribution of the detection frame coordinates predicted by the first model is a Gaussian distribution, the coordinate variance corresponding to the detection frame coordinates is the variance corresponding to the Gaussian distribution.

[0085] In addition, it should be noted that, generally speaking, a detection frame needs to be determined by multiple detection frame coordinates, and accordingly, each detection frame coordinate also has its corresponding coordinate variance. For example, a rectangular detection frame can be determined by the coordinates of four vertices, and accordingly, each vertex has a corresponding coordinate variance.

[0086] Step 250 , obtaining pseudo-annotation information corresponding to the annotation information according to the third information and the fourth information corresponding to each processed image.

[0087] The pseudo-annotation information includes pseudo-label information and pseudo-position information. The pseudo-label information is used to indicate the category of the detection target, and the pseudo-position information is used to indicate the position of the detection target in the sample image.

[0088] In some embodiments, step 250 includes at least one of the following sub-steps 252 to 256 .

[0089] Sub-step 252, removing duplicate information for the same detection target from the third information and the fourth information corresponding to each processed image, to obtain deduplicated third information and deduplicated fourth information corresponding to each processed image.

[0090] Repeated information for the same detection target refers to information that repeatedly indicates the position and category of the same detection target in a single processed image. Exemplarily, the fourth information corresponding to an image may include the detection frame coordinates of multiple detection frames for the same detection target. Correspondingly, in the fourth information after deduplication, for each detection target in the image, there is only one detection frame coordinate corresponding to each detection frame.

[0091] In some embodiments, a non-maximum suppression (NMS) algorithm may be used to remove the above-mentioned duplicate information.

[0092] Sub-step 254, clustering the deduplicated third information and the deduplicated fourth information corresponding to each processed image according to the deduplicated fourth information corresponding to each processed image, to obtain at least one information cluster.

[0093] The information cluster includes the deduplicated third information and the deduplicated fourth information corresponding to the same detection target. In other words, the deduplicated third information and the deduplicated fourth information included in one information cluster are both information for the same detection target.

[0094] Clustering refers to dividing a data set into different classes or clusters according to a specific standard, so that the similarity of data objects in the same cluster is as large as possible, and the difference of data objects in different clusters is as large as possible. In other words, after clustering, data of the same class is gathered together as much as possible, and data of different classes is separated as much as possible.

[0095] In some embodiments, the fourth information includes the detection box coordinates, and sub-step 254 may include the following steps:

[0096] 1. Determine any one of the detection frames corresponding to each processed image as a central detection frame corresponding to a first information cluster in at least one information cluster.

[0097] That is, one is selected from all the detection frames corresponding to the processed images to be determined as the central detection frame. The third information after deduplication and the fourth information after deduplication corresponding to the central detection frame are the clustering centers of the first information clustering.

[0098] 2. Calculate the overlap between the detection frame corresponding to each processed image and the central detection frame.

[0099] That is to say, the overlap between all the detection frames and the central detection frame is calculated respectively. In some embodiments, the overlap can be represented by an intersection-over-union ratio.

[0100] In addition, it should be noted that in the embodiment of the present application, when calculating the degree of overlap, all detection frame coordinates are uniformly calculated under the coordinate scale of the sample image. For example, when the sample image is flipped or enlarged, the detection frame coordinates therein will not be flipped or enlarged accordingly.

[0101] 3. Determine the deduplicated third information and the deduplicated fourth information corresponding to the detection frames whose overlap degree is greater than the first threshold as the information included in the first information cluster.

[0102] The first threshold may be a preset value. For example, when the overlap degree is represented by the intersection-over-union ratio, the first threshold may be 0.3.

[0103] 4. Repeat the above method to cluster the remaining deduplicated third information and deduplicated fourth information until all deduplicated third information and deduplicated fourth information are clustered (all detection boxes corresponding to each processed image are clustered).

[0104] The above method clusters information according to the overlap of detection frames, and can effectively cluster information for the same detection target into one category, thereby obtaining information clusters corresponding to each detection target.

[0105] Sub-step 256, determining pseudo-labeled information according to a target information cluster that matches the labeled information in at least one information cluster.

[0106] Information clustering and annotation information matching means that the detection target targeted by the information clustering is the detection target corresponding to the annotation information. In some embodiments, the center frame coordinates of each cluster can be matched with the position information in the annotation information. This application does not limit the specific method of matching information clustering with annotation information.

[0107] In some embodiments, since information clustering is performed based on the overlap of detection frames, the information of the detection target corresponding to the annotation information may be clustered into multiple clusters. At this time, when determining the pseudo-label information, the above-mentioned multiple clusters can be regarded as target information clusters, and when determining the pseudo-position information, one of the above-mentioned multiple clusters can be selected as the target information cluster.

[0108] Through steps such as deduplication and clustering, the above method can obtain information clusters for different detection targets, and these information clusters can be matched with different labeling information corresponding to different detection targets. Furthermore, based on the information in the target information cluster, pseudo-labeling information corresponding to the labeling information can be calculated, so that in the process of training the target detection model, the pseudo-labeling information can be used to combat the noise that may exist in the labeling information, thereby achieving better model performance.

[0109] In some embodiments, sub-step 256 includes at least one of the following steps:

[0110] 1. According to the third information after deduplication in the target information clustering, the pseudo label information is obtained.

[0111] In some embodiments, the third information includes the first vector, and the pseudo label information can be obtained by the following method:

[0112] The mean of each first vector included in the target information cluster is calculated to obtain a second vector; and the second vector is determined as pseudo label information.

[0113] That is to say, by taking the average of all the first vectors included in the target information cluster in each dimension, the pseudo label information can be obtained.

[0114] The above method takes the mean of the first vector and comprehensively utilizes all the third information after deduplication in the target information clustering, so that the probability that the final pseudo-label information deviates from the true value is reduced, thereby reducing the label noise.

[0115] 2. According to the fourth information after deduplication in the target information clustering, the pseudo position information is obtained.

[0116] In some embodiments, the fourth information includes the detection frame coordinates and the coordinate variance corresponding to the detection frame coordinates. This step includes the following sub-steps:

[0117] (1) calculating at least one intermediate probability according to the intersection-over-union ratio of at least one detection box corresponding to the target information cluster and the central detection box corresponding to the target information cluster;

[0118] In some embodiments, for the i-th detection box corresponding to the target information cluster, its corresponding intermediate probability p i It can be calculated by the following formula: Among them, IoU(b i ,b) is the intersection-over-combination ratio of the i-th detection box corresponding to the target information cluster and the center detection box, σ t is a hyperparameter.

[0119] (2) At least one pseudo coordinate is calculated based on the coordinates of each detection frame included in the target information clustering, the coordinate variance corresponding to each detection frame coordinate, and at least one intermediate probability.

[0120] The pseudo coordinates are the coordinate values ​​of the pseudo frame, which is also used to indicate the position of the detection target in the sample image. The number of pseudo coordinates is related to the shape of the detection frame and the pseudo frame. For example, if both the detection frame and the pseudo frame are rectangular frames, the number of pseudo coordinates is 4.

[0121] In some embodiments, the pseudo coordinate x may be calculated using the following formula: Among them, x i is the detection box coordinates of the i-th detection box corresponding to the target information cluster, σ x,i For x i The corresponding coordinate variance. In addition, it should be noted that x and x i They correspond to each other in position. Taking the detection frame and pseudo frame as rectangular frames as an example, if x is the coordinate of the upper left vertex of the pseudo frame, then x i It should also be the coordinate of the upper left vertex of the detection box.

[0122] In addition, it should be noted that the above calculation process of the pseudo-coordinate x is actually the process of finding the mean of a discrete probability distribution. In this discrete probability distribution, x i The corresponding probability is Therefore, in some embodiments, the variance of the discrete probability distribution may be determined as the variance corresponding to the pseudo-coordinates. For its specific role, please refer to the following embodiments.

[0123] (3) Determine at least one pseudo coordinate as pseudo position information.

[0124] By determining the pseudo position information using the above method, full use is made of the detection frame coordinates, coordinate variance and other information included in the fourth information after deduplication in the target information clustering, so that the probability that the final pseudo position information deviates from the true value is reduced, thereby reducing the border noise.

[0125] In summary, through the above method, pseudo label information and pseudo position information with low noise can be obtained, thereby obtaining pseudo annotation information with low noise.

[0126] Step 260, adjusting the parameters of the target detection model according to the first information, the second information, the labeling information and the pseudo-labeling information to obtain a trained target detection model.

[0127] In some embodiments, step 260 includes at least one of the following sub-steps 262 to 272 .

[0128] Sub-step 262, based on the first information and the label information, calculate the label loss, where the label loss is used to characterize the difference between the first information and the label information.

[0129] Sub-step 264, based on the first information and the pseudo-label information, calculate the pseudo-label loss, where the pseudo-label loss is used to characterize the difference between the first information and the pseudo-label information.

[0130] In some embodiments, the first information includes a third vector, and both the label information and the pseudo-label information are represented in vector form. At this time, the Focal Loss between the third vector and the label information can be determined as the label loss, and the Focal Loss between the third vector and the pseudo-label information can be determined as the pseudo-label loss. The above is only an exemplary description. For label loss and pseudo-label loss, other types of loss functions (such as cross entropy loss) can also be used for calculation, and this application does not limit this.

[0131] Sub-step 266, calculating the position loss based on the second information and the position information, where the position loss is used to characterize the difference between the second information and the position information.

[0132] Sub-step 268, based on the second information and the pseudo position information, calculate the pseudo position loss, where the pseudo position loss is used to characterize the difference between the second information and the pseudo position information.

[0133] In some embodiments, the second information, the position information, and the pseudo-position information all include coordinates indicating the position of the detection target in the sample image (such as detection frame coordinates), and the position loss and pseudo-position loss can be determined by calculating L1Loss based on these coordinates. The above is only an exemplary description, and other types of loss functions (such as Dice Loss, Glou Loss) can also be used to calculate the position loss and pseudo-position loss, which is not limited in this application.

[0134] In sub-step 270, the label loss, pseudo-label loss, position loss and pseudo-position loss are weightedly summed to obtain a total loss, which is used to measure the performance of the target detection model.

[0135] In some embodiments, the pseudo-label information includes the second vector, the pseudo-position information includes at least one pseudo-coordinate, and sub-step 270 includes at least one of the following steps:

[0136] 1. According to the second vector, determine a first weight, and the first weight is used to perform a weighted summation of the label loss and the pseudo-label loss.

[0137] In some embodiments, the negative value of the information entropy of the probability distribution corresponding to the second vector is determined as the first weight.

[0138] The first weight Among them, ε is the information entropy, is the second vector.

[0139] Furthermore, the label loss and pseudo-label loss can be weighted and summed by the following formula to obtain the total label loss: Among them, L cls is the label loss, is the pseudo position loss.

[0140] The information entropy of the probability distribution corresponding to the second vector can reflect the uncertainty of the pseudo-label information. The larger the information entropy, the higher the uncertainty of the pseudo-label information, and the lower the reliability of the corresponding pseudo-label loss. Therefore, by assigning weights to the label loss and pseudo-label loss through the above method, a better total label loss can be obtained, thereby improving the performance of the target detection model.

[0141] 2. Determine a second weight corresponding to at least one pseudo-coordinate based on the coordinate variance corresponding to at least one pseudo-coordinate, wherein the second weight is used to perform weighted summation of the position loss and the pseudo-position loss, and the coordinate variance corresponding to the pseudo-coordinate is obtained from the fourth information.

[0142] For the specific method of calculating the coordinate variance corresponding to the pseudo-coordinate, please refer to the above embodiment. Alternatively, in some embodiments, the coordinate variance corresponding to the pseudo-coordinate can also be obtained directly based on the coordinates corresponding to the pseudo-coordinates in the fourth information (or the coordinate variances corresponding to these coordinates), and this application does not limit this.

[0143] In some embodiments, for a first pseudo coordinate in at least one pseudo coordinate, a negative value of the information entropy of a Gaussian distribution with a coordinate variance corresponding to the first pseudo coordinate as a standard deviation is determined as a second weight corresponding to the first pseudo coordinate.

[0144] That is, the second weight corresponding to the first pseudo coordinate in, is the Gaussian distribution mentioned above.

[0145] Below, the first pseudo coordinate is an upper left coordinate of a rectangular pseudo frame for exemplary description. If the pseudo position loss is obtained based on the first pseudo coordinate and the upper left coordinate of the detection frame in the second information, Based on the upper left coordinate of the annotation box in the position information and the upper left coordinate of the detection box in the second information, the position loss is L bbox , then for the upper left coordinate, this point Furthermore, for the entire rectangular frame, the total position loss can be determined based on the weighted position losses corresponding to the four points of the upper left, lower left, upper right, and lower right. For example, the total position loss can be equal to the sum of the weighted position losses corresponding to the four points of the upper left, lower left, upper right, and lower right.

[0146] It should be noted that no matter what shape of detection frame is used or what method of representing the position is used, as long as the second information, position information and pseudo-position information indicate the detection target through coordinates, the above method can be used to obtain the total position loss.

[0147] The information entropy of the Gaussian distribution can reflect the uncertainty of the pseudo-position information. The larger the information entropy, the higher the uncertainty of the pseudo-position information, and the lower the reliability of the corresponding pseudo-position loss. Therefore, by assigning weights to the position loss and pseudo-position loss through the above method, a better total position loss can be obtained, thereby improving the performance of the target detection model.

[0148] 3. According to the first weight and the second weight corresponding to at least one pseudo-coordinate, the label loss, pseudo-label loss, position loss and pseudo-position loss are weightedly summed to obtain the total loss.

[0149] The total loss is the sum of the above total label loss and total position loss.

[0150] By performing weighted summation to obtain the total loss through the above method, weights can be flexibly assigned to the above various losses according to the specific circumstances of the pseudo-label information and pseudo-position information, so as to obtain a better total loss and thus improve the performance of the target detection model.

[0151] Sub-step 272, with the purpose of minimizing the total loss, adjust the parameters of the target detection model to obtain a trained target detection model.

[0152] In some embodiments, based on the total loss, the parameters of the target detection model are adjusted by gradient back propagation to obtain a trained target detection model.

[0153] The above method obtains the total loss by weighted summing the label loss, pseudo-label loss, position loss and pseudo-position loss, and adjusts the parameters of the target detection model according to the total loss. In the case where there is noise in the annotation information and the label loss and position loss are inaccurate, the model can also adjust the parameters with the help of pseudo-label loss and pseudo-position loss, thereby improving the noise resistance of the model training process.

[0154] In some embodiments, the parameters of the first model are adjusted according to the parameters of the trained target detection model to obtain an adjusted first model.

[0155] In some embodiments, according to the parameters of the trained target detection model, the parameters of the first model are adjusted by an EMA (Exponential Moving Average) algorithm to obtain an adjusted first model.

[0156] The adjusted first model is used to participate in the next round of training process of the target detection model. Therefore, by continuously updating the parameters of the first model through the above method, the prediction results of the first model can be made more and more accurate as the training process progresses, thereby obtaining pseudo-labeling information with less noise, thereby reducing the degree to which the training process of the target detection model is affected by the noise of the labeled data.

[0157] The technical solution provided by the embodiment of the present application is obtained by obtaining a sample image including at least one detection target and annotation information that can indicate the category and position of the detection target in the sample image. In addition, the target detection model and the first model are respectively made to predict the category and position of the detection target according to the sample image and the processed image obtained by data enhancement processing of the sample image. Afterwards, according to the prediction result of the first model, pseudo-annotation information that can also be used to indicate the category and position of the detection target in the sample image is obtained. Finally, according to the above-mentioned annotation information, pseudo-annotation information and the prediction result of the target detection model, the parameters of the target detection model are adjusted to obtain the trained target detection model. By training the target detection model by the above-mentioned method, the influence of the noise of the category error and the position error of the detection target in the annotation information on the model training process can be effectively weakened, and there is no need to screen the annotation information, and all training resources can be fully utilized, thereby improving the model performance. At the same time, the method has no specific requirements for the structure of the target detection model and has good generalization.

[0158] Please refer to Figure 3 , which shows a schematic diagram of a complete solution of a method for training a target detection model provided by an embodiment of the present application. The solution may include the following steps.

[0159] 1. Obtain a sample image and perform data enhancement processing on the sample image to obtain at least one processed image.

[0160] 2. Input the sample image into the target detection model 30 to obtain the output of the target detection model 30 (first information and second information), input at least one processed image into the first model 40 to obtain the output of the first model 40 (third information and fourth information corresponding to each processed image).

[0161] 3. According to the output of the first model 40, pseudo-labeling information is obtained.

[0162] 4. According to the output of the target detection model 30, the label loss, position loss, pseudo-label loss and pseudo-position loss are calculated. According to the output of the first model 40 (and the pseudo-label information), the first weight α corresponding to the label loss and the pseudo-label loss, and the second weight β corresponding to the position loss and the pseudo-position loss are determined. According to the first weight α and the second weight β, the above-mentioned label loss, position loss, pseudo-label loss and pseudo-position loss are weighted and summed to obtain the total loss. According to the total loss, the parameters of the target detection model are adjusted, and according to the updated parameters of the target detection model, the EMA algorithm is used to adjust the parameters of the first model accordingly.

[0163] For the contents not described in detail in this embodiment, please refer to the above embodiments and will not be repeated here.

[0164] Please refer to Figure 4 , which shows a schematic diagram of the test results of using different training methods for the same model.

[0165] The target detection model and the reference model have the same structure and initial parameters, and are trained using the same data set. The number of training rounds is 12, and the test index is MAP (Mean Average Precision). The higher the index, the better the target detection performance of the model. At the same time, the target detection model is trained using the training method provided in the embodiment of the present application, and the reference model is trained using the existing training method in the field of target detection.

[0166] It can be seen that the target detection model can obtain higher MAP indicators at each stage of training. The test results prove the effectiveness of the technical solution of the present application in resisting the noise of the labeled data.

[0167] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0168] Please refer to Figure 5 , which shows a block diagram of a training device for a target detection model provided by an embodiment of the present application. The device has the function of implementing the training method of the above-mentioned target detection model, and the function can be implemented by hardware, or by hardware executing corresponding software. The device can be a computer device, or it can be set in a computer device. The device 500 may include: an acquisition module 510, an enhancement module 520, a target detection module 530, a first detection module 540, a pseudo-annotation generation module 550 and an adjustment module 560.

[0169] The acquisition module 510 is used to acquire at least one sample image and the annotation information corresponding to the sample image; wherein the sample image includes at least one detection target, and the annotation information includes label information and position information of the detection target, the label information is used to indicate the category of the detection target, and the position information is used to indicate the position of the detection target in the sample image.

[0170] The enhancement module 520 is used to perform data enhancement processing on the sample image to obtain at least one processed image.

[0171] The target detection module 530 is used to obtain first information and second information based on the sample image through a target detection model; wherein the first information is used to indicate the prediction result of the target detection model for the category of the detection target, and the second information is used to indicate the prediction result of the target detection model for the position of the detection target in the sample image.

[0172] The first detection module 540 is used to obtain third information and fourth information corresponding to each processed image according to the at least one processed image through the first model; wherein the third information is used to indicate the prediction result of the first model for the category of the detection target, and the fourth information is used to indicate the prediction result of the first model for the position of the detection target in the processed image.

[0173] The pseudo-annotation generation module 550 is used to obtain pseudo-annotation information corresponding to the annotation information according to the third information and the fourth information corresponding to each of the processed images, wherein the pseudo-annotation information includes pseudo-label information and pseudo-position information. The pseudo-label information is used to indicate the category of the detection target, and the pseudo-position information is used to indicate the position of the detection target in the sample image.

[0174] The adjustment module 560 is used to adjust the parameters of the target detection model according to the first information, the second information, the labeling information and the pseudo-labeling information to obtain a trained target detection model.

[0175] In some embodiments, the pseudo-label generation module 550 includes: a deduplication submodule, a clustering submodule, and a pseudo-label determination submodule ( Figure 5 not shown).

[0176] The deduplication submodule is used to remove duplicate information for the same detection target in the third information and the fourth information corresponding to each of the processed images, so as to obtain the deduplicated third information and the deduplicated fourth information corresponding to each of the processed images.

[0177] A clustering submodule is used to cluster the deduplicated third information and the deduplicated fourth information corresponding to each of the processed images according to the deduplicated fourth information corresponding to each of the processed images, so as to obtain at least one information cluster, wherein the information cluster includes the deduplicated third information and the deduplicated fourth information corresponding to the same detection target.

[0178] The pseudo-label determination submodule is used to determine the pseudo-label information according to a target information cluster matching the label information in the at least one information cluster.

[0179] In some embodiments, the fourth information includes detection box coordinates, where the detection box is used to indicate the position of the detection target in the processed image.

[0180] The clustering submodule is used to determine any one of the detection frames corresponding to each of the processed images as a central detection frame corresponding to the first information cluster in the at least one information cluster; calculate the overlap between the detection frames corresponding to each of the processed images and the central detection frame; and determine the deduplicated third information and the deduplicated fourth information corresponding to the detection frames whose overlap is greater than a first threshold as the information included in the first information cluster.

[0181] In some embodiments, the pseudo-annotation determination submodule includes: a pseudo-label determination unit and a pseudo-position determination unit.

[0182] The pseudo label determination unit is used to obtain the pseudo label information according to the third information after duplication in the target information clustering.

[0183] The pseudo position determining unit is used to obtain the pseudo position information according to the fourth information after duplication in the target information cluster.

[0184] In some embodiments, the third information includes a first vector, and the first vector is used to represent the prediction result of the first model on the probability distribution of the category of the detection target.

[0185] The pseudo label determination unit is used to calculate the mean of each of the first vectors included in the target information cluster to obtain a second vector; and determine the second vector as the pseudo label information.

[0186] In some embodiments, the fourth information includes detection box coordinates and coordinate variances corresponding to the detection box coordinates, the detection box is used to indicate the position of the detection target in the processed image, and the coordinate variances corresponding to the detection box coordinates are used to indicate the degree of discreteness of the probability distribution of the detection box coordinates predicted by the first model.

[0187] A pseudo-position determination unit is used to calculate at least one intermediate probability based on the intersection and union ratio of at least one detection frame corresponding to the target information cluster and the central detection frame corresponding to the target information cluster; calculate at least one pseudo-coordinate based on the coordinates of each detection frame included in the target information cluster, the coordinate variance corresponding to each detection frame coordinate and the at least one intermediate probability; and determine the at least one pseudo-coordinate as the pseudo-position information.

[0188] In some embodiments, the adjustment module 560 includes: a loss calculation submodule, a weighted summation submodule, and a parameter adjustment submodule ( Figure 5 not shown).

[0189] A loss calculation submodule is used to calculate a label loss based on the first information and the label information, wherein the label loss is used to characterize the difference between the first information and the label information; calculate a pseudo-label loss based on the first information and the pseudo-label information, wherein the pseudo-label loss is used to characterize the difference between the first information and the pseudo-label information; calculate a position loss based on the second information and the position information, wherein the position loss is used to characterize the difference between the second information and the position information; calculate a pseudo-position loss based on the second information and the pseudo-position information, wherein the pseudo-position loss is used to characterize the difference between the second information and the pseudo-position information.

[0190] The weighted summation submodule is used to perform weighted summation on the label loss, the pseudo-label loss, the position loss and the pseudo-position loss to obtain a total loss, and the total loss is used to measure the performance of the target detection model.

[0191] The parameter adjustment submodule is used to adjust the parameters of the target detection model with the purpose of minimizing the total loss to obtain the trained target detection model.

[0192] In some embodiments, the pseudo label information includes a second vector, where the second vector is used to characterize the probability distribution of the category of the detection target, and the pseudo position information includes at least one pseudo coordinate.

[0193] The weighted summation submodule includes: a first weight determination unit, a second weight determination unit and a total loss determination unit.

[0194] A first weight determination unit is used to determine a first weight according to the second vector, where the first weight is used to perform a weighted summation of the label loss and the pseudo-label loss.

[0195] A second weight determination unit is used to determine the second weight corresponding to the at least one pseudo-coordinate according to the coordinate variance corresponding to the at least one pseudo-coordinate, the second weight is used to perform weighted summation of the position loss and the pseudo-position loss, and the coordinate variance corresponding to the pseudo-coordinate is obtained from the fourth information.

[0196] A total loss determination unit is used to perform weighted summation of the label loss, the pseudo-label loss, the position loss and the pseudo-position loss according to the first weight and the second weight corresponding to the at least one pseudo-coordinate, so as to obtain the total loss.

[0197] In some embodiments, the first weight determination unit is used to determine the negative value of the information entropy of the probability distribution corresponding to the second vector as the first weight.

[0198] In some embodiments, the second weight determination unit is used to determine, for a first pseudo coordinate among the at least one pseudo coordinate, a negative value of the information entropy of a Gaussian distribution with a coordinate variance corresponding to the first pseudo coordinate as a standard deviation as a second weight corresponding to the first pseudo coordinate.

[0199] In some embodiments, the data enhancement processing includes at least one of the following: changing the size of the sample image, flipping the sample image, and changing the luminosity of the sample image.

[0200] In some embodiments, the adjustment module 560 is further used to adjust the parameters of the first model according to the parameters of the trained target detection model to obtain an adjusted first model.

[0201] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0202] Please refer to Figure 6 , which exemplarily shows a structural block diagram of a computer device provided by an embodiment of the present application.

[0203] Typically, the computer device 600 includes a processor 601 and a memory 602 .

[0204] The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may also include an AI processor, which is used to process computing operations related to machine learning.

[0205] The memory 602 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 602 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 stores a computer program, which is loaded and executed by the processor 601 to implement the training method of the above-mentioned target detection model.

[0206] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the computer device 600, and the computer device 600 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.

[0207] In some embodiments, a computer-readable storage medium is also provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned target detection model training method.

[0208] Optionally, the computer readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives) or optical disks, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0209] In some embodiments, a computer program product is also provided, which includes a computer program stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the above-mentioned target detection model training method.

[0210] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0211] The above are merely exemplary embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training a target detection model, characterized in that: The method comprises: Acquire at least one sample image and annotation information corresponding to the sample image; wherein the sample image includes at least one detection target, and the annotation information includes label information and position information of the detection target, the label information is used to indicate the category of the detection target, and the position information is used to indicate the position of the detection target in the sample image; Performing data enhancement processing on the sample image to obtain at least one processed image; Obtaining first information and second information according to the sample image through a target detection model; wherein the first information is used to indicate a prediction result of the target detection model on the category of the detection target, and the second information is used to indicate a prediction result of the target detection model on the position of the detection target in the sample image; Obtaining third information and fourth information respectively corresponding to each of the processed images according to the at least one processed image through the first model; wherein the third information is used to indicate the prediction result of the first model for the category of the detection target, and the fourth information is used to indicate the prediction result of the first model for the position of the detection target in the processed image, and the fourth information includes detection box coordinates and coordinate variances corresponding to the detection box coordinates, the detection box is used to indicate the position of the detection target in the processed image, and the coordinate variances corresponding to the detection box coordinates are used to indicate the discrete degree of the probability distribution of the detection box coordinates predicted by the first model; Removing repeated information for the same detection target from the third information and the fourth information corresponding to each of the processed images, to obtain deduplicated third information and deduplicated fourth information corresponding to each of the processed images; Clustering the deduplicated third information and the deduplicated fourth information respectively corresponding to each of the processed images according to the deduplicated fourth information respectively corresponding to each of the processed images to obtain at least one information cluster, wherein the information cluster includes the deduplicated third information and the deduplicated fourth information corresponding to the same detection target; Obtaining pseudo label information according to the third information after duplication in the target information cluster, wherein the pseudo label information is used to indicate the category of the detection target, and the target information cluster is an information cluster in the at least one information cluster that matches the annotation information; Calculating at least one intermediate probability according to an intersection-over-union ratio of at least one detection box corresponding to the target information cluster and a central detection box corresponding to the target information cluster; Calculate at least one pseudo coordinate according to each of the detection frame coordinates included in the target information cluster, the coordinate variance corresponding to each of the detection frame coordinates, and the at least one intermediate probability; Determine the at least one pseudo coordinate as pseudo position information, where the pseudo position information is used to indicate the position of the detection target in the sample image; According to the first information, the second information, the annotation information, the pseudo-label information and the pseudo-position information, the parameters of the target detection model are adjusted to obtain a trained target detection model.

2. The method according to claim 1, characterized in that The step of clustering the deduplicated third information and the deduplicated fourth information respectively corresponding to each of the processed images according to the deduplicated fourth information respectively corresponding to each of the processed images to obtain at least one information cluster comprises: Determine any one of the detection frames corresponding to each of the processed images as a central detection frame corresponding to a first information cluster in the at least one information cluster; Calculating the overlap between the detection frames corresponding to the processed images and the central detection frame; The deduplicated third information and the deduplicated fourth information corresponding to the detection frames whose overlap degree is greater than the first threshold are determined as the information included in the first information cluster.

3. The method according to claim 1, characterized in that The third information includes a first vector, where the first vector is used to represent a prediction result of the first model on the probability distribution of the category of the detection target; The step of obtaining pseudo label information according to the third information after deduplication in the target information clustering includes: Calculating the mean of each of the first vectors included in the target information cluster to obtain a second vector; The second vector is determined as the pseudo label information.

4. The method according to claim 1, characterized in that The step of adjusting the parameters of the target detection model according to the first information, the second information, the annotation information, the pseudo label information, and the pseudo position information to obtain a trained target detection model includes: Calculate a label loss based on the first information and the label information, where the label loss is used to characterize the difference between the first information and the label information; Based on the first information and the pseudo label information, calculating a pseudo label loss, where the pseudo label loss is used to characterize the difference between the first information and the pseudo label information; Calculate a position loss based on the second information and the position information, where the position loss is used to characterize a difference between the second information and the position information; Calculating a pseudo position loss based on the second information and the pseudo position information, wherein the pseudo position loss is used to characterize a difference between the second information and the pseudo position information; Performing a weighted summation of the label loss, the pseudo-label loss, the position loss, and the pseudo-position loss to obtain a total loss, wherein the total loss is used to measure the performance of the target detection model; With the purpose of minimizing the total loss, the parameters of the target detection model are adjusted to obtain the trained target detection model.

5. The method according to claim 4, characterized in that The pseudo label information includes a second vector, the second vector is used to characterize the probability distribution of the category of the detection target, and the pseudo position information includes at least one pseudo coordinate; The weighted summing of the label loss, the pseudo-label loss, the position loss and the pseudo-position loss to obtain the total loss includes: Determine a first weight according to the second vector, where the first weight is used to perform a weighted summation of the label loss and the pseudo label loss; Determine, according to the coordinate variance respectively corresponding to the at least one pseudo coordinate, a second weight respectively corresponding to the at least one pseudo coordinate, wherein the second weight is used to perform a weighted summation of the position loss and the pseudo position loss, and the coordinate variance corresponding to the pseudo coordinate is obtained from the fourth information; According to the first weight and the second weight corresponding to the at least one pseudo coordinate respectively, the label loss, the pseudo label loss, the position loss and the pseudo position loss are weightedly summed to obtain the total loss.

6. The method according to claim 5, characterized in that The determining a first weight according to the second vector includes: The negative value of the information entropy of the probability distribution corresponding to the second vector is determined as the first weight.

7. The method according to claim 5, characterized in that The determining, according to the coordinate variance respectively corresponding to the at least one pseudo-coordinate, the second weight respectively corresponding to the at least one pseudo-coordinate comprises: For a first pseudo coordinate of the at least one pseudo coordinate, a negative value of the information entropy of a Gaussian distribution with a coordinate variance corresponding to the first pseudo coordinate as a standard deviation is determined as a second weight corresponding to the first pseudo coordinate.

8. The method according to claim 1, characterized in that The data enhancement processing includes at least one of the following: changing the size of the sample image, flipping the sample image, and changing the luminosity of the sample image.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: According to the parameters of the trained target detection model, the parameters of the first model are adjusted to obtain an adjusted first model.

10. A training device for a target detection model, characterized in that: The device comprises: An acquisition module, configured to acquire at least one sample image and annotation information corresponding to the sample image; wherein the sample image includes at least one detection target, and the annotation information includes label information and position information of the detection target, wherein the label information is used to indicate the category of the detection target, and the position information is used to indicate the position of the detection target in the sample image; An enhancement module, used for performing data enhancement processing on the sample image to obtain at least one processed image; A target detection module, configured to obtain first information and second information according to the sample image through a target detection model; wherein the first information is used to indicate a prediction result of the target detection model on the category of the detection target, and the second information is used to indicate a prediction result of the target detection model on the position of the detection target in the sample image; A first detection module, configured to obtain third information and fourth information respectively corresponding to each of the processed images according to the at least one processed image through a first model; wherein the third information is used to indicate the prediction result of the first model for the category of the detection target, and the fourth information is used to indicate the prediction result of the first model for the position of the detection target in the processed image, and the fourth information includes a detection box coordinate and a coordinate variance corresponding to the detection box coordinate, the detection box is used to indicate the position of the detection target in the processed image, and the coordinate variance corresponding to the detection box coordinate is used to indicate the discrete degree of the probability distribution of the detection box coordinate predicted by the first model; The pseudo-annotation generation module is used to remove duplicate information for the same detection target in the third information and the fourth information corresponding to each of the processed images, and obtain the deduplicated third information and the deduplicated fourth information corresponding to each of the processed images; cluster the deduplicated third information and the deduplicated fourth information corresponding to each of the processed images according to the deduplicated fourth information corresponding to each of the processed images, and obtain at least one information cluster, wherein the information cluster includes the deduplicated third information and the deduplicated fourth information corresponding to the same detection target; obtain pseudo-label information according to the deduplicated third information in the target information cluster, The pseudo-label information is used to indicate the category of the detection target, and the target information cluster is an information cluster in the at least one information cluster that matches the annotation information; at least one intermediate probability is calculated based on the intersection-and-union ratio of at least one detection frame corresponding to the target information cluster and the central detection frame corresponding to the target information cluster; at least one pseudo-coordinate is calculated based on the coordinates of each detection frame included in the target information cluster, the coordinate variance corresponding to each detection frame coordinate, and the at least one intermediate probability; the at least one pseudo-coordinate is determined as pseudo-position information, and the pseudo-position information is used to indicate the position of the detection target in the sample image; An adjustment module is used to adjust the parameters of the target detection model according to the first information, the second information, the annotation information, the pseudo-label information and the pseudo-position information to obtain a trained target detection model.

11. A computer device, characterized in that: The computer device comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product comprises a computer program, wherein the computer program is stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the method according to any one of claims 1 to 9.

Images