Method, device, equipment and storage medium for training target detection model

Abstract

The present application provides a method, apparatus, device and storage medium for training a target detection model, which can be applied to the fields of artificial intelligence or smart medical care, etc., to solve the problem of low detection accuracy and detection reliability of the trained target detection model. The method at least includes: using an auxiliary detection model to predict first pseudo-labels that conform to fine-grained types corresponding to multiple sampling labels respectively; using an auxiliary detection model to predict second pseudo-labels that conform to fine-grained types corresponding to multiple transformed images based on multiple transformed images and corresponding second sample labels; training the target detection model based on the obtained multiple first pseudo-labels and multiple second pseudo-labels. By making full use of the coarse-grained type labels, the detection accuracy and detection reliability of the trained target detection model are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for training a target detection model. Background Art

[0002] With the continuous development of technology, more and more devices can provide target detection services through trained target detection models. The target detection service can be used to determine the location of the target in the image.

[0003] For example, the device can use a trained target detection model to locate abnormalities in chest X-ray (CXR) images to assist in the clinical diagnosis of various chest diseases.

[0004] In related technologies, since fine-grained sample annotations of sample images need to be manually annotated by experienced professionals, the method for obtaining a trained target detection model is usually to perform semi-supervised training on the target detection model based on a small number of sample images with associated fine-grained sample annotations and a large number of sample images without associated sample annotations.

[0005] However, since the number of sample images associated with fine-grained sample annotations in the related technology is small, while the number of sample images without associated sample annotations is large, the training based on sample images without associated sample annotations is usually only carried out based on the training process based on sample images associated with fine-grained sample annotations, resulting in less reliable information to refer to during the training process, and thus the detection accuracy of the trained target detection model is low.

[0006] It can be seen that the training method adopted under the relevant technology cannot guarantee the detection accuracy and reliability of the trained target detection model. Summary of the Invention

[0007] The embodiments of the present application provide a method, apparatus, computer device, and storage medium for training a target detection model, which are used to solve the problem of low detection accuracy and detection reliability of the trained target detection model.

[0008] In a first aspect, a method for training an object detection model is provided, comprising:

[0009] Acquire a first sample image set and a second sample image set, wherein each first sample image is associated with a first sample annotation of a fine-grained type, and each second sample image is associated with a second sample annotation of a coarse-grained type, wherein the annotation accuracy of the fine-grained type is higher than that of the coarse-grained type;

[0010] Based on the first sample image set and the second sample image set, multiple rounds of iterative training are performed to output a trained object detection model, wherein each round of iteration includes:

[0011] Sampling the first sample annotation of the first sample image to obtain a plurality of sample annotations that conform to the coarse-grained type, and using the auxiliary detection model to predict first pseudo annotations that conform to the fine-grained type corresponding to each of the plurality of sample annotations;

[0012] Using the auxiliary detection model, based on multiple transformed images corresponding to the second sample image and the corresponding second sample annotations, respectively predict second pseudo annotations that correspond to each of the multiple transformed images and conform to the fine-grained type;

[0013] The target detection model is trained based on the obtained multiple first pseudo-annotations and multiple second pseudo-annotations.

[0014] In a second aspect, a target detection method is provided, wherein the target detection model is obtained based on the method for training the target detection model described in the first aspect and includes:

[0015] Obtain the image to be detected;

[0016] Using the target detection model, extracting features from the image to be detected to obtain image features;

[0017] Using the target detection model, based on the image features, when it is determined that the image to be detected contains a specified target, a fine-grained type of predicted annotation output by the trained target detection model is obtained, wherein the fine-grained type characterization indicates that the annotation accuracy of the predicted annotation reaches a preset threshold, and the predicted annotation is used to mark the position of the specified target in the image to be detected.

[0018] In a third aspect, a device for training an object detection model is provided, comprising:

[0019] An acquisition module is configured to acquire a first sample image set and a second sample image set, wherein each first sample image is associated with a first sample annotation of a fine-grained type, and each second sample image is associated with a second sample annotation of a coarse-grained type, wherein the annotation accuracy of the fine-grained type is higher than that of the coarse-grained type;

[0020] A processing module is configured to perform multiple rounds of iterative training based on the first sample image set and the second sample image set, and output a trained object detection model, wherein each round of iteration includes:

[0021] The processing module is specifically configured to: perform sampling processing on the first sample annotation of the first sample image to obtain a plurality of sample annotations that conform to the coarse-grained type, and use the auxiliary detection model to predict first pseudo annotations that conform to the fine-grained type corresponding to each of the plurality of sample annotations;

[0022] The processing module is further configured to: use the auxiliary detection model to predict, based on the plurality of transformed images corresponding to the second sample image and the corresponding second sample annotations, second pseudo annotations that correspond to the plurality of transformed images and conform to the fine-grained type;

[0023] The processing module is further configured to train a target detection model based on the obtained multiple first pseudo labels and multiple second pseudo labels.

[0024] Optionally, the processing module is specifically configured to:

[0025] Determining a sampling consistency loss of the auxiliary detection model based on the obtained multiple first pseudo-annotations, wherein the sampling consistency loss represents: consistency of predictions based on the multiple sampling annotations;

[0026] Determining a transformation consistency loss of the auxiliary detection model based on the obtained plurality of second pseudo-annotations, wherein the transformation consistency loss represents: consistency of predictions based on a plurality of transformed images;

[0027] Determining a prediction loss of the auxiliary detection model based on the multiple first pseudo-labels and the first sample label, wherein the prediction loss represents: an accuracy of the predicted pseudo-labels;

[0028] The object detection model is trained based on the obtained sampling consistency loss, transformation consistency loss and prediction loss.

[0029] Optionally, the processing module is specifically configured to:

[0030] Adjusting model parameters of the auxiliary detection model based on the obtained sampling consistency loss, transformation consistency loss, and prediction loss;

[0031] Using the auxiliary detection model, predicting that the second sample image conforms to a third pseudo-annotation of the fine-grained type;

[0032] Using the object detection model, predict a first predicted label for the first sample image, and predict a second predicted label for the second sample image;

[0033] Based on the obtained error between the first predicted label and the first sample label, and the error between the second predicted label and the third pseudo label, the model parameters of the object detection model are adjusted.

[0034] Optionally, the processing module is specifically configured to:

[0035] respectively determining an error between every two first pseudo-labels in the plurality of first pseudo-labels to obtain a sampling consistency error;

[0036] Based on the obtained weighted average of the sampling consistency errors, the sampling consistency loss of the auxiliary detection model is determined.

[0037] Optionally, the second pseudo-annotation is in the form of an image having the same image size as the corresponding transformed image; then the processing module is specifically configured to:

[0038] According to the image transformation strategy of the transformed image corresponding to one second pseudo-annotation, performing image transformation on the other second pseudo-annotations respectively to obtain corresponding transformed annotations;

[0039] Determining the error between each two annotations in each annotation including the second pseudo annotation and each obtained transformed annotation to obtain each transformation consistency error;

[0040] Based on the obtained weighted average of the transformation consistency errors, the transformation consistency loss of the auxiliary detection model is determined.

[0041] Optionally, the processing module is specifically configured to:

[0042] Determining prediction errors between the plurality of first pseudo annotations and the first sample annotation respectively;

[0043] The prediction loss of the auxiliary detection model is determined based on a weighted average of the obtained multiple prediction errors.

[0044] Optionally, the processing module is specifically configured to:

[0045] Based on the image flipping strategy, performing image flipping on the second sample image to obtain a first transformed image;

[0046] Based on the image masking strategy, using at least one specified covering pattern, performing image masking on the second sample image to obtain a second transformed image;

[0047] The auxiliary detection model is used to predict second pseudo labels of the first transformed image and the second transformed image that meet the fine-grained type based on the first transformed image, the second transformed image, and the second sample label.

[0048] Optionally, the processing module is further configured to:

[0049] After performing multiple rounds of iterative training based on the first sample image set and the second sample image set and outputting a trained object detection model, obtaining an image to be detected;

[0050] Using the target detection model, extracting features from the image to be detected to obtain image features;

[0051] When the target detection model is used to determine, based on the image features, that the image to be detected contains a specified target, a fine-grained type of prediction annotation output by the trained target detection model is obtained, wherein the prediction annotation is used to mark the position of the specified target in the image to be detected.

[0052] In a fourth aspect, a target detection device is provided, wherein the target detection model is obtained based on the method for training the target detection model described in the first aspect, including:

[0053] Acquisition module: used to acquire the image to be detected;

[0054] Processing module: used to use the target detection model to extract features of the image to be detected to obtain image features;

[0055] The processing module is also used to use the target detection model to determine, based on the image features, that the image to be detected contains a specified target, and obtain the fine-grained type of predicted annotation output by the trained target detection model, wherein the fine-grained type characterization indicates that the annotation accuracy of the predicted annotation reaches a preset threshold, and the predicted annotation is used to mark the position of the specified target in the image to be detected.

[0056] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method as described in the first aspect or the second aspect.

[0057] According to a sixth aspect, a computer device is provided, comprising:

[0058] a memory for storing program instructions;

[0059] The processor is configured to call the program instructions stored in the memory and execute the method according to the first aspect or the second aspect according to the obtained program instructions.

[0060] In a seventh aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method as described in the first aspect or the second aspect.

[0061] In an embodiment of the present application, two sample image sets, a first sample image set and a second sample image set, are obtained, the first sample images are associated with first sample annotations of a fine-grained type, and the second sample images are associated with second sample annotations of a coarse-grained type, and the annotation accuracy of the fine-grained type is higher than that of the coarse-grained type. Therefore, when the target detection model is trained based on the first sample image set and the second sample image set, the problem of low training accuracy will not be caused by too few images of the first sample images associated with the fine-grained first sample annotations, resulting in too little training data available for training the target detection model; and the problem of low training accuracy will not be caused by a large training error when training the target detection model due to the low annotation accuracy of the second sample annotations associated with the coarse-grained type. To a certain extent, the detection accuracy and detection reliability of the trained target detection model are improved.

[0062] Furthermore, in one round of iteration, an auxiliary detection model is used to predict the first pseudo-annotations that conform to the fine-grained type corresponding to each of the multiple sampling annotations obtained by sampling the first sample annotation. Thus, by ensuring the consistency of the multiple first pseudo-annotations, when the target detection model is trained based on the obtained first pseudo-annotations, the multiple annotations of the coarse-grained type marked at different positions can be accurately determined as the annotations of the fine-grained type belonging to the same position, and there will be no situation where a fine-grained type annotation is determined for each of the multiple annotations of the coarse-grained type at different positions, which misleads the training process of the target detection model. Therefore, the trained target detection model can accurately determine the fine-grained type annotation, which improves the detection accuracy and reliability of the trained target detection model to a certain extent.

[0063] Furthermore, in one round of iteration, an auxiliary detection model is used to predict the second pseudo-annotations that conform to the fine-grained type corresponding to each of the multiple transformed images based on the multiple transformed images corresponding to the second sample image and the corresponding second sample annotations. Thus, by ensuring the consistency of multiple second pseudo-annotations, when the target detection model is trained based on the obtained second pseudo-annotations, the coarse-grained type annotations located at the boundary of the fine-grained type annotations can be accurately determined as annotations belonging to the fine-grained type, and there will be no situation where the coarse-grained type annotations located at the boundary of the fine-grained type annotations are mistakenly identified as annotations belonging to other fine-grained types, which will mislead the training process of the target detection model. Therefore, the trained target detection model can accurately determine the fine-grained type annotations, which improves the detection accuracy and reliability of the trained target detection model to a certain extent.

[0064] The target detection model is trained from multiple perspectives of multiple first pseudo-labels and multiple second pseudo-labels, thereby further improving the detection accuracy and detection reliability of the trained target detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1A Schematic diagram of the application field of the target detection model provided in the embodiment of the present application;

[0066] Figure 1B An application scenario of the method for training an object detection model provided in an embodiment of the present application;

[0067] Figure 2 A flowchart of a method for training an object detection model provided in an embodiment of the present application is shown in FIG1 ;

[0068] Figure 3A Schematic diagram 1 of a principle of a method for training an object detection model provided in an embodiment of the present application;

[0069] Figure 3B A schematic diagram of the principle of the method for training the target detection model provided in the embodiment of the present application Figure 2 ;

[0070] Figure 4A A third schematic diagram of a principle of a method for training an object detection model provided in an embodiment of the present application;

[0071] Figure 4B A fourth schematic diagram of a principle of a method for training an object detection model provided in an embodiment of the present application;

[0072] Figure 4C Schematic diagram 5 of a principle of a method for training an object detection model provided in an embodiment of the present application;

[0073] Figure 5A A schematic diagram of the principle of the method for training the target detection model provided in the embodiment of the present application Figure 6 ;

[0074] Figure 5B A schematic diagram of the principle of the method for training the target detection model provided in the embodiment of the present application Figure 7 ;

[0075] Figure 5C A schematic diagram of the principle of the method for training the target detection model provided in the embodiment of the present application Figure 8 ;

[0076] Figure 5D A schematic diagram of the principle of the method for training the target detection model provided in the embodiment of the present application Figure 9 ;

[0077] Figure 5E A schematic diagram of the principle of the method for training the target detection model provided in the embodiment of the present application Figure 10 ;

[0078] Figure 6 A schematic diagram of the principle of the method for training the target detection model provided in the embodiment of the present application Figure 10 one;

[0079] Figure 7 A flow chart of a method for training a target detection model provided in an embodiment of the present application Figure 2 ;

[0080] Figure 8 A schematic diagram of the structure of an apparatus for training an object detection model provided in an embodiment of the present application;

[0081] Figure 9 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application;

[0082] Figure 10 A schematic diagram of the structure of a device for training a target detection model or a target detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0084] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0085] (1) Semi-supervised training:

[0086] Semi-supervised training is a training method for machine learning that completes model training given a portion of fully labeled data and a large amount of unlabeled data.

[0087] The embodiments of the present application relate to the field of artificial intelligence (AI) and are designed based on computer vision (CV) technology and machine learning (ML) technology. They can be applied to cloud computing, smart transportation, smart agriculture, smart healthcare, maps and other fields.

[0088] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that studies the design principles and implementation methods of various machines, attempting to understand the essence of intelligence and produce new intelligent machines that can respond in ways similar to human intelligence, enabling them to perceive, reason, and make decisions.

[0089] Artificial intelligence is a comprehensive discipline covering a wide range of fields, including both hardware and software technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation interaction systems, mechatronics, and other technologies. The software technologies of artificial intelligence mainly include computer vision technology, speech processing technology, natural language processing technology, machine learning / deep learning, autonomous driving, smart transportation, and other major directions. With the development and progress of artificial intelligence, artificial intelligence has been able to be researched and applied in many fields, such as common smart homes, smart customer service, virtual assistants, smart speakers, smart marketing, smart wearable devices, unmanned driving, autonomous driving, drones, robots, smart healthcare, Internet of Vehicles, autonomous driving, smart transportation, and other fields. It is believed that with the further development of future technology, artificial intelligence will be applied in more fields and play an increasingly important role. The solutions provided in the embodiments of this application involve technologies such as deep learning and augmented reality of artificial intelligence, which are further illustrated by the following embodiments.

[0090] Computer vision is the study of how machines can "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects. Further image processing is performed to transform the computer's image into an image more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, and smart transportation. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0091] Machine learning is a multidisciplinary interdisciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers can simulate human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structures, and enable computers to continuously improve their own performance.

[0092] Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. At the core of machine learning is deep learning, a technology that enables machine learning. Machine learning generally includes techniques such as deep learning, reinforcement learning, transfer learning, inductive learning, artificial neural networks, and self-learning. Deep learning includes technologies such as convolutional neural networks (CNNs), deep belief networks, recurrent neural networks, autoencoders, and generative adversarial networks.

[0093] It should be noted that in the embodiments of the present application, data related to the first sample image set, the second sample image set or the images to be detected are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0094] The following is a brief introduction to the application areas of the method for training target detection models provided in the embodiments of the present application.

[0095] With the continuous development of technology, more and more devices can provide target detection services through trained target detection models. The target detection service can be used to determine the location of the target in the image.

[0096] For example, the device can use a trained target detection model to locate abnormalities in chest X-ray (CXR) images to assist in the clinical diagnosis of various chest diseases.

[0097] Since the fine-grained sample annotation of the sample image needs to be obtained manually by experienced professionals, for example, please refer to Figure 1A For chest X-ray images, rectangular boxes can be used to mark abnormal areas in the image to locate the abnormality. This rectangular box annotation involves the length, width, and position of the rectangle, making it more fine-grained than point-based annotation. This requires manual annotation by experienced professionals.

[0098] Therefore, in related technologies, the method for obtaining a trained target detection model is usually to perform semi-supervised training on the target detection model based on a small number of sample images with associated fine-grained sample annotations and a large number of sample images without associated sample annotations.

[0099] However, since the number of sample images associated with fine-grained sample annotations in the related technology is small, while the number of sample images without associated sample annotations is large, the training based on sample images without associated sample annotations is usually only carried out based on the training process based on sample images associated with fine-grained sample annotations, resulting in less reliable information to refer to during the training process, and thus the detection accuracy of the trained target detection model is low.

[0100] It can be seen that the training method adopted under the relevant technology cannot guarantee the detection accuracy and reliability of the trained target detection model.

[0101] In order to solve the problem of low detection accuracy and reliability of the trained target detection model, the present application proposes a method for training a target detection model. In this method, after obtaining a first sample image set and a second sample image set, multiple rounds of iterative training are performed based on the first sample image set and the second sample image set, and a trained target detection model is output. Each first sample image is associated with a first sample annotation of a fine-grained type, and each second sample image is associated with a second sample annotation of a coarse-grained type. The annotation accuracy of the fine-grained type is higher than that of the coarse-grained type. Each round of iteration includes:

[0102] The first sample annotation of the first sample image is sampled to obtain multiple sample annotations that conform to the coarse-grained type. The auxiliary detection model is used to predict first pseudo annotations that conform to the fine-grained type corresponding to each of the multiple sample annotations. The auxiliary detection model is used to predict second pseudo annotations that conform to the fine-grained type corresponding to each of the multiple transformed images corresponding to the second sample image and the corresponding second sample annotations. The object detection model is trained based on the obtained multiple first pseudo annotations and multiple second pseudo annotations.

[0103] In an embodiment of the present application, two sample image sets, a first sample image set and a second sample image set, are obtained, the first sample images are associated with first sample annotations of a fine-grained type, and the second sample images are associated with second sample annotations of a coarse-grained type, and the annotation accuracy of the fine-grained type is higher than that of the coarse-grained type. Therefore, when the target detection model is trained based on the first sample image set and the second sample image set, the problem of low training accuracy will not be caused by too few images of the first sample images associated with the fine-grained first sample annotations, resulting in too little training data available for training the target detection model; and the problem of low training accuracy will not be caused by a large training error when training the target detection model due to the low annotation accuracy of the second sample annotations associated with the coarse-grained type. To a certain extent, the detection accuracy and detection reliability of the trained target detection model are improved.

[0104] Furthermore, in one round of iteration, an auxiliary detection model is used to predict the first pseudo-annotations that conform to the fine-grained type corresponding to each of the multiple sampling annotations obtained by sampling the first sample annotation. Thus, by ensuring the consistency of the multiple first pseudo-annotations, when the target detection model is trained based on the obtained first pseudo-annotations, the multiple annotations of the coarse-grained type marked at different positions can be accurately determined as the annotations of the fine-grained type belonging to the same position, and there will be no situation where a fine-grained type annotation is determined for each of the multiple annotations of the coarse-grained type at different positions, which misleads the training process of the target detection model. Therefore, the trained target detection model can accurately determine the fine-grained type annotation, which improves the detection accuracy and reliability of the trained target detection model to a certain extent.

[0105] Furthermore, in one round of iteration, an auxiliary detection model is used to predict the second pseudo-annotations that conform to the fine-grained type corresponding to each of the multiple transformed images based on the multiple transformed images corresponding to the second sample image and the corresponding second sample annotations. Thus, by ensuring the consistency of multiple second pseudo-annotations, when the target detection model is trained based on the obtained second pseudo-annotations, the coarse-grained type annotations located at the boundary of the fine-grained type annotations can be accurately determined as annotations belonging to the fine-grained type, and there will be no situation where the coarse-grained type annotations located at the boundary of the fine-grained type annotations are mistakenly identified as annotations belonging to other fine-grained types, which will mislead the training process of the target detection model. Therefore, the trained target detection model can accurately determine the fine-grained type annotations, which improves the detection accuracy and reliability of the trained target detection model to a certain extent.

[0106] The target detection model is trained from multiple perspectives of multiple first pseudo-labels and multiple second pseudo-labels, thereby further improving the detection accuracy and detection reliability of the trained target detection model.

[0107] The following describes the application scenarios of the method for training target detection models provided in this application.

[0108] Please refer to Figure 1B , which is a schematic diagram of an application scenario of the method for training a target detection model provided in this application. The application scenario includes a client 101 and a server 102. The client 101 and the server 102 can communicate with each other. The communication method can be to use wired communication technology, for example, to communicate via a network cable or a serial cable; or to use wireless communication technology, for example, to communicate via Bluetooth or wireless fidelity (WIFI) and other technologies, without specific limitation.

[0109] The client 101 generally refers to a device that can provide the first sample image set, the second sample image set, or can use the trained target detection model to the server 102, such as a terminal device, a third-party application that the terminal device can access, or a web page that the terminal device can access. Terminal devices include but are not limited to mobile phones, computers, smart medical devices, smart home appliances, vehicle-mounted terminals or aircraft, etc. The server 102 generally refers to a device that can train or use the target detection model, such as a terminal device or a server, etc. The server includes but is not limited to a cloud server, a local server or an associated third-party server, etc. Both the client 101 and the server 102 can use cloud computing to reduce the occupation of local computing resources; cloud storage can also be used to reduce the occupation of local storage resources.

[0110] As an embodiment, the client 101 and the server 102 can be the same device, and there is no specific limitation. In the embodiment of the present application, the client 101 and the server 102 are different devices as an example for introduction.

[0111] The following is based on Figure 1B , taking the server 102 as the server and the server as the main body, the method for training the target detection model provided by the embodiment of the present application is specifically introduced. Please refer to Figure 2 , which is a flow chart of the method for training a target detection model provided in an embodiment of the present application.

[0112] S201: Acquire a first sample image set and a second sample image set.

[0113] The server may receive the first sample image set and the second sample image set sent by the client, or read the first sample image set and the second sample image set from other devices, or obtain the first sample image set and the second sample image set by downloading data from network resources, etc., without specific limitation.

[0114] For example, the network resources include two CXR datasets, the RSNA dataset and the VinDr-CXR dataset. The RSNA dataset contains 26,684 chest X-ray images, and the VinDr-CXR dataset contains 15,000 chest X-ray images. The server can divide the image data in the two CXR datasets into a training dataset and a test dataset in proportion. The training dataset is used to train the target detection model, and the test dataset is used to test the trained target detection model. For the divided training dataset, a part of it can be used as the first sample image set and the other part as the second sample image set in proportion. For example, the first sample image set accounts for 5%, 10%, 20%, 30%, 40% or 50% of the training dataset, etc., without specific restrictions.

[0115] Each first sample image is associated with a first sample annotation of the fine-grained type, and each second sample image is associated with a second sample annotation of the coarse-grained type. The annotation accuracy of the fine-grained type is higher than that of the coarse-grained type. For example, the first sample annotation of the fine-grained type is a sample annotation in the form of a rectangular box, and the second sample annotation of the coarse-grained type is a sample annotation in the form of a dot, a graffiti, or a line. For CXR images, the sample annotation in the form of a rectangular box can mark the entire area where the lesion occurs, while the sample annotation in the form of a dot can only mark a certain location of the local lesion. Therefore, the sample annotation in the form of a rectangular box has a higher annotation accuracy than the sample annotation in the form of a dot.

[0116] The first sample annotation may be obtained through manual annotation, and the second sample annotation may be obtained through manual annotation or device annotation, etc., without specific limitation.

[0117] S202 , performing multiple rounds of iterative training based on the first sample image set and the second sample image set, and outputting a trained object detection model.

[0118] After obtaining the first and second sample image sets, the server can perform multiple rounds of iterative training on the target detection model to be trained based on the first and second sample image sets, and output a trained target detection model. During the training of the target detection model, an auxiliary detection model can be combined with the training to help the target detection model more quickly learn target detection capabilities.

[0119] The following takes the process of one round of iterative training as an example. The process of each round of iterative training is similar and will not be described in detail here. Please refer to S203 to S205.

[0120] S203 , performing sampling processing on the first sample annotation of the first sample image to obtain multiple sampling annotations that conform to the coarse-grained type, and using the auxiliary detection model to respectively predict first pseudo annotations that conform to the fine-grained type corresponding to each of the multiple sampling annotations.

[0121] When performing a round of iterative training based on a first sample image and a second sample image, since the first sample annotation of the first sample image is a sample annotation that conforms to the fine-grained type, in order to ensure the prediction consistency of the auxiliary detection model, the first sample annotation of the first sample image can be sampled first to obtain multiple sampling annotations that conform to the coarse-grained type, and then the auxiliary detection model is used to predict the first pseudo-annotations that conform to the fine-grained type corresponding to each of the multiple sampling annotations based on the first sample image and the multiple sampling annotations. The multiple first pseudo-annotations obtained should be consistent with each other. Therefore, the auxiliary detection model can be trained through multiple first pseudo-annotations to improve the prediction accuracy of the auxiliary detection model.

[0122] For example, if the fine-grained type is a rectangular box and the coarse-grained type is a point, then sampling the first sample annotation in the rectangular box form can randomly obtain multiple point-based sample annotations located within the rectangular box. When using the auxiliary detection model to predict the first pseudo-annotations in the form of rectangular boxes corresponding to each of the multiple sample annotations, the predicted rectangular boxes of each first pseudo-annotation should have similar lengths and widths and be located in similar positions.

[0123] Please refer to Figure 3A The first sample image includes a first sample annotation in the form of a rectangular frame. The first sample annotation in the form of a rectangular frame is sampled to obtain three sample annotations in the form of points, including a first sample annotation, a second sample annotation, and a third sample annotation. The first sample annotation is located near the upper edge of the first sample annotation in the form of a rectangular frame, the second sample annotation is located near the center of the first sample annotation in the form of a rectangular frame, and the third sample annotation is located near the left edge of the first sample annotation in the form of a rectangular frame.

[0124] Please refer to Figure 3B After sampling the first sample annotation in the form of a rectangular box, three sample annotations in the form of points are obtained, including a first sample annotation, a second sample annotation, and a third sample annotation. Using the auxiliary detection model, based on the first sample image and the three sample annotations, first pseudo annotations in the form of rectangular boxes corresponding to the three sample annotations are predicted, and first pseudo annotations A, B, and C are obtained, respectively. The rectangular boxes of the first pseudo annotations A, B, and C have similar lengths and widths and are located in similar positions.

[0125] As an embodiment, the prediction process of the auxiliary detection model can be expressed by a formula. Please refer to formula (1). It can be seen that the auxiliary detection model is a model used to predict the corresponding sample annotations that meet the fine-grained type based on the sample image and the corresponding sample annotations that meet the coarse-grained type.

[0126]

[0127] Among them, X p The representation position is (x, y), and the sample label c that meets the coarse-grained type represents X i The i-th second sample image, Characterize predicted sample annotations that conform to fine-grained types.

[0128] S204 , using the auxiliary detection model, based on the multiple transformed images corresponding to the second sample image and the corresponding second sample annotations, respectively predict second pseudo annotations that conform to the fine-grained type corresponding to each of the multiple transformed images.

[0129] To further utilize the second sample image associated with the coarse-grained second sample annotation, the server can perform image transformations on the second sample image based on multiple image transformation strategies to obtain multiple transformed images. Using the auxiliary detection model, based on the obtained multiple transformed images and the second sample annotation, the corresponding second pseudo-annotations conforming to the fine-grained type should be consistent with each other. Therefore, training the auxiliary detection model with multiple second pseudo-annotations can improve the auxiliary detection model's prediction accuracy.

[0130] In one embodiment, when the image transformation strategy includes an image flipping strategy and an image masking strategy, the server may perform an image flipping on the second sample image based on the image flipping strategy to obtain a first transformed image. Based on the image masking strategy, the server may perform an image masking on the second sample image using at least one specified overlay pattern to obtain a second transformed image. Furthermore, the server may utilize an auxiliary detection model to predict, based on the first transformed image, the second transformed image, and the second sample annotation, second pseudo-annotations that conform to the fine-grained type for the first transformed image and the second transformed image, respectively.

[0131] The image flip strategy is used to flip the second sample image left to right, up to down, or rotate it 90°, 180°, or 270° around the center of the image. There is no specific restriction. Figure 4A (1) is a second sample image, after being flipped left and right, please refer to Figure 4A (2) is a first transformed image.

[0132] The image masking strategy is used to use at least one specified overlay pattern to cover the second sample image to achieve image masking, so that the position covered by the specified overlay pattern in the obtained second transformed image cannot display the content displayed at that position in the original second sample image. Figure 4B (1) is a second sample image. After being covered by three rectangular graphics, please refer to Figure 4B (2) is a second transformed image.

[0133] Please refer to Figure 4C After the second sample image is flipped left to right, the first transformed image obtained forms a mirror image relationship with the second sample image.

[0134] After randomly masking the second sample image using three rectangular figures, the obtained second transformed image contains three dot-rectangular blank areas.

[0135] After predicting second pseudo-annotations in the form of rectangular frames corresponding to the first transformed image and the second transformed image using the auxiliary detection model based on the second sample annotation, the first transformed image, and the second transformed image, a second pseudo-annotation A and a second pseudo-annotation B are obtained. The second pseudo-annotation A and the second pseudo-annotation B form a mirror image relationship. When predicting the second pseudo-annotation in the form of a rectangular frame corresponding to the first transformed image based on the second sample annotation and the first transformed image, the second sample annotation is a left-right flipped second sample annotation.

[0136] S205 : Training the target detection model based on the obtained multiple first pseudo-labels and multiple second pseudo-labels.

[0137] After obtaining the plurality of first pseudo-annotations and the plurality of second pseudo-annotations, the server may train the target detection model based on the plurality of first pseudo-annotations and the plurality of second pseudo-annotations. Since the target detection model training process is performed in conjunction with the auxiliary detection model, the auxiliary detection model may be trained once based on the plurality of first pseudo-annotations and the plurality of second pseudo-annotations, and then the target detection model may be trained once based on the trained auxiliary detection model, and so on, without further limitation.

[0138] As an embodiment, when training a target detection model based on the obtained multiple first pseudo-annotations and multiple second pseudo-annotations, the sampling consistency loss of the auxiliary detection model can be determined based on the obtained multiple first pseudo-annotations, wherein the sampling consistency loss represents: the consistency of the prediction based on the multiple sampling annotations. Based on the obtained multiple second pseudo-annotations, the transformation consistency loss of the auxiliary detection model is determined, wherein the transformation consistency loss represents: the consistency of the prediction based on multiple transformed images. Based on the multiple first pseudo-annotations and the first sample annotations, the prediction loss of the auxiliary detection model is determined, wherein the prediction loss represents: the accuracy of the predicted pseudo-annotations. The target detection model is trained based on the obtained sampling consistency loss, transformation consistency loss and prediction loss.

[0139] The following is a detailed introduction to the process of obtaining sampling consistency loss, transformation consistency loss and prediction loss.

[0140] Sampling consistency loss:

[0141] The error between each two first pseudo-labels in the plurality of first pseudo-labels is determined to obtain sampling consistency errors, and the sampling consistency loss of the auxiliary detection model is determined based on a weighted average of the obtained sampling consistency errors.

[0142] For example, multiple first pseudo-labels include two first pseudo-labels, where one first pseudo-label is predicted based on the first sample image and the first sampling label, and the other first pseudo-label is predicted based on the first sample image and the second sampling label. Then the sampling consistency loss L m , please refer to formula (2).

[0143]

[0144] in, Characterization based on the first sample image X i and the first sample annotation The first pseudo-label predicted, Characterization based on the first sample image X i and the second sampling annotation The first pseudo-label predicted, ‖·‖2 represents the calculation of the L2 norm. L m Narrowed the gap between the two first pseudo-annotations.

[0145] Transformation consistency loss:

[0146] When the second pseudo-annotation is in the form of an image with the same image size as the corresponding transformed image, the remaining second pseudo-annotations are transformed according to the image transformation strategy for the transformed image corresponding to the second pseudo-annotation to obtain corresponding transformed annotations. The error between each pair of annotations, including the second pseudo-annotation and the obtained transformed annotations, is determined to obtain transformation consistency errors. The transformation consistency loss of the auxiliary detection model is determined based on the weighted average of the obtained transformation consistency errors.

[0147] Take the example of two second pseudo-labels as the multiple second pseudo-labels, where one second pseudo-label is based on the first transformed image obtained by flipping the second sample image left and right, and the second sample label prediction; the other second pseudo-label is based on the second transformed image obtained by randomly covering the second sample image with three rectangular images, and the second sample label prediction. Then the transformation consistency loss L c , please refer to formula (3).

[0148]

[0149] in, Characterize the second sample image X i The first transformed image obtained by left-right flipping, Characterize the second sample label X P After flipping left and right, Represents the second pseudo annotation predicted based on the first transformed image and the second sample annotation after left-right flipping, M(Xi ) represents the second sample image X i The second transformed image obtained by random covering, X P +σ represents the sum of the second sample annotation and the additional noise term. The additional noise term can be evenly distributed in the image ratio range of [-0.05, 0.05]. The additional noise term is used to prevent overfitting. d (X P +σ,M(X i )) represents the second pseudo annotation predicted based on the second transformed image and the second sample annotation, and T(·) represents the left-right flip transformation. L c The mechanism is that the auxiliary detection model should be able to predict consistent fine-grained type annotations under different image transformations.

[0150] Prediction loss:

[0151] Prediction errors between the plurality of first pseudo-annotations and the first sample annotation are determined respectively, and a prediction loss of the auxiliary detection model is determined based on a weighted average of the plurality of obtained prediction errors.

[0152] Taking the case where the multiple first pseudo-labels are two first pseudo-labels, the server first determines the prediction error between the first pseudo-label A and the first sample label, and then determines the prediction error between the first pseudo-label B and the first sample label. The weighted average of the two prediction errors can be used as the prediction loss L of the auxiliary detection model. box Etc., no specific restrictions.

[0153] As an embodiment, after obtaining the sampling consistency loss, transformation consistency loss, and prediction loss, the model parameters of the auxiliary detection model can be adjusted based on the obtained sampling consistency loss, transformation consistency loss, and prediction loss. For example, the sampling consistency loss, transformation consistency loss, and prediction loss are weighted and summed to obtain the training loss of the auxiliary detection model. When it is determined that the training loss of the auxiliary detection model reaches the training target, the model parameters of the auxiliary detection model are not adjusted, and the target detection model is trained directly based on the auxiliary detection model. When it is determined that the training loss of the auxiliary detection model does not meet the training target, the model parameters of the auxiliary detection model are adjusted, and the target detection model is trained based on the auxiliary detection model after the model parameters are adjusted, etc., and there is no specific limitation.

[0154] When training the target detection model based on the auxiliary detection model, the auxiliary detection model can be used to predict the third pseudo-label of the second sample image that meets the fine-grained type. The target detection model is used to predict the first predicted label of the first sample image and the second predicted label of the second sample image. Based on the error between the obtained first predicted label and the first sample label, and the error between the second predicted label and the third pseudo-label, the model parameters of the target detection model are adjusted. For example, the error between the obtained first predicted label and the first sample label, and the error between the second predicted label and the third pseudo-label are weightedly summed to obtain the training loss of the target detection model. When it is determined that the training loss of the target detection model reaches the training target, the trained target detection model is output. When it is determined that the training loss of the target detection model does not meet the training target, the model parameters of the target detection model are adjusted, and the next round of iterative training is entered, etc. There are no specific restrictions.

[0155] Please refer to formula (4).

[0156]

[0157] in, Representing the target detection model F s Based on the second sample image The second prediction annotation of the prediction, Representing the target detection model F s Based on the first sample image The first predicted annotation, F d (X P , X i ) represents the third pseudo-label predicted by the auxiliary detection model, Y represents the first sample label, L o Represent a loss function.

[0158] Below, taking CXR images as an example, the method for training the target detection model provided in the embodiment of the present application is introduced with an example of one round of iterative training.

[0159] Please refer to Figure 5A (1) is a possible first sample image, please refer to Figure 5A (2) is a possible second sample image. The first sample of the first sample image is marked as a rectangular frame, and the second sample of the second sample image is marked as a dot.

[0160] The auxiliary detection model includes a label feature extraction network, an image feature extraction network and a prediction network. The label feature extraction network is used to extract the features of labels such as sample labels, the image feature extraction network is used to extract the features of images such as sample images, and the prediction network is used to predict pseudo-labels in the form of rectangular boxes based on the obtained features.

[0161] Please refer to Figure 5B After sampling the first sample annotation of the first sample image, two sample annotations in the form of points are obtained. Taking one sample annotation as an example, the image feature extraction network in the auxiliary detection model is used to extract image features from the first sample image to obtain a first image feature. The annotation feature extraction network in the auxiliary detection model is then used to extract annotation features from the sample annotation to obtain a first annotation feature. The prediction network in the auxiliary detection model is used to predict a first pseudo-annotation in the form of a rectangular box corresponding to the sample annotation based on the first image feature and the first annotation feature.

[0162] After obtaining the first pseudo-annotations corresponding to the two sample annotations, the sampling consistency loss of the auxiliary detection model is determined based on the error between the two first pseudo-annotations and the error between the two first pseudo-annotations and the first sample annotation.

[0163] Please refer to Figure 5C After the second sample image is flipped left to right, a first transformed image is obtained, and the corresponding second sample annotation is also flipped left to right. After the second sample image is masked, a second transformed image is obtained.

[0164] The image feature extraction network in the auxiliary detection model is used to extract image features from the first transformed image and the second transformed image, respectively, to obtain second image features and third image features, respectively. The annotation feature extraction network in the auxiliary detection model is used to extract annotation features from the second sample annotation after left-right flipping and the second sample annotation, respectively, to obtain second annotation features and third annotation features, respectively.

[0165] The prediction network in the auxiliary detection model is used to predict a second pseudo label of the first transformed image based on the second image feature and the second annotation feature, and to predict a second pseudo label of the second transformed image based on the third image feature and the third annotation feature.

[0166] Since the first transformed image is obtained by left-right flipping, the second pseudo-annotation of the first transformed image is in a mirror image relationship with the second pseudo-annotation of the second transformed image. Therefore, the second pseudo-annotation of the first transformed image can be flipped left-right. Thus, the transformation consistency loss of the auxiliary detection model can be determined based on the error between the second pseudo-annotation after left-right flipping of the first transformed image and the second pseudo-annotation of the second transformed image.

[0167] Therefore, based on the sampling consistency loss and transformation consistency loss, the auxiliary detection model can be trained for one round to obtain the auxiliary detection model after one round of training. Figure 5DThe second sample image is input into the trained auxiliary detection model. The image feature extraction network in the trained auxiliary detection model is used to extract image features from the second sample image to obtain a fourth image feature. The annotation feature extraction network in the trained auxiliary detection model is also used to extract annotation features from the second sample annotation to obtain a fourth annotation feature. The prediction network in the trained auxiliary detection model can predict a third pseudo-annotation in the form of a rectangular box for the second sample image based on the obtained fourth image feature and the fourth annotation feature.

[0168] Please refer to Figure 5E After obtaining the third pseudo-annotation in the form of a rectangular frame for the second sample image, the first sample image and the second sample image can be respectively input into the object detection model to predict a first predicted annotation for the first sample image and a second predicted annotation for the second sample image. Based on the error between the first predicted annotation and the first sample annotation, and the error between the second predicted annotation and the third pseudo-annotation, a training loss of the object detection model is determined, and the object detection model is trained for one round based on the training loss of the object detection model.

[0169] Please refer to Figure 6 , which is a visualization result of the trained auxiliary detection model predicting the sample annotation in the form of a rectangular box based on multiple point-form sample annotations, as well as a visualization result of the model in other related technologies predicting the sample annotation in the form of a rectangular box based on multiple point-form sample annotations.

[0170] Please refer to Figure 6 (1), the left half contains three point-form annotations. When the model in the related art is used to predict the rectangular box-form sample annotations based on the point-form sample annotations, the obtained rectangular box-form sample annotations can only focus on the corresponding point-form sample annotations, but cannot focus on other nearby point-form sample annotations. As a result, the positions of the three obtained rectangular box-form sample annotations are quite different, that is, the positions of the rectangular box-form sample annotations with the thickest lines, the rectangular box-form sample annotations with the thinnest lines, and the rectangular box-form sample annotations with moderate line thickness are quite different. At the same time, the position difference with the manual annotation is also quite large, that is, the position of the rectangular box-form sample annotations is quite different from the position of the dotted box annotations. The same is true for the right half.

[0171] Please refer to Figure 6 (2) When the trained auxiliary detection model obtained by the method of the embodiment of the present application predicts the sample annotations in the form of rectangular boxes based on the sample annotations in the form of points, the position difference of the sample annotations in the form of rectangular boxes obtained is small and almost overlaps. At the same time, the position difference with the manual annotation is also small. In fact, the sample annotations in the form of rectangular boxes are smaller in range than the manually labeled rectangles and more accurate.

[0172] Please refer to Figure 6 (3) Containing three point-shaped annotations that are closer to each other, when the model in the related technology is used to predict the rectangular box-shaped sample annotations based on the point-shaped sample annotations, the obtained rectangular box-shaped sample annotations still have a large position difference, and the position difference between them and the dotted box annotations is also large.

[0173] Please refer to Figure 6 (4) When the trained auxiliary detection model obtained by the method of the embodiment of the present application predicts the sample annotations in the form of rectangular boxes based on the sample annotations in the form of points, the position difference of the sample annotations in the form of rectangular boxes obtained is small and almost overlaps. At the same time, the position difference with the manual annotation is also small.

[0174] In the embodiments of this application, robust point-based annotations generate rectangular pseudo-annotations with higher accuracy, resulting in higher detection accuracy and reliability for the target detection model trained based on the pseudo-annotations. This approach fully utilizes a large amount of coarse-grained annotated data, significantly alleviating the problem of insufficient annotated data and achieving training results comparable to those of fully supervised algorithms, thereby improving the detection accuracy and reliability of the trained target detection model.

[0175] Based on the same inventive concept, the embodiment of the present application provides a target detection method, which uses the target detection model trained by the aforementioned method of training the target detection model to perform target detection. Please refer to Figure 7 , which is a flowchart of the target detection method.

[0176] S701: Acquire an image to be detected.

[0177] S702: Using the target detection model, extract features from the image to be detected to obtain image features.

[0178] S703, using the target detection model, based on image features, when determining that the image to be detected contains the specified target, obtain the fine-grained type prediction annotation output by the trained target detection model, wherein the fine-grained type characterization prediction annotation has a labeling accuracy reaching a preset threshold, and the prediction annotation is used to mark the position of the specified target in the image to be detected.

[0179] Based on the same inventive concept, the present application embodiment provides a device for training a target detection model, which can realize the functions corresponding to the aforementioned method for training a target detection model. Figure 8 , the device includes an acquisition module 801 and a processing module 802, wherein:

[0180] Acquisition module 801: used to acquire a first sample image set and a second sample image set, wherein each first sample image is associated with a first sample annotation of a fine-grained type, and each second sample image is associated with a second sample annotation of a coarse-grained type, wherein the annotation accuracy of the fine-grained type is higher than that of the coarse-grained type;

[0181] Processing module 802 is configured to perform multiple rounds of iterative training based on the first sample image set and the second sample image set, and output a trained object detection model, wherein each round of iteration includes:

[0182] The processing module 802 is specifically configured to: perform sampling processing on the first sample annotation of the first sample image to obtain a plurality of sample annotations that conform to the coarse-grained type, and use the auxiliary detection model to predict first pseudo annotations that conform to the fine-grained type corresponding to each of the plurality of sample annotations;

[0183] The processing module 802 is further configured to: use the auxiliary detection model to predict, based on the multiple transformed images corresponding to the second sample image and the corresponding second sample annotation, second pseudo annotations that conform to the fine-grained type corresponding to each of the multiple transformed images;

[0184] The processing module 802 is further configured to train the target detection model based on the obtained multiple first pseudo labels and multiple second pseudo labels.

[0185] In a possible embodiment, the processing module 802 is specifically configured to:

[0186] Determining a sampling consistency loss of the auxiliary detection model based on the obtained multiple first pseudo-annotations, wherein the sampling consistency loss represents: consistency of predictions based on the multiple sampling annotations;

[0187] Determining a transformation consistency loss of the auxiliary detection model based on the obtained multiple second pseudo-annotations, wherein the transformation consistency loss represents: consistency of prediction based on multiple transformed images;

[0188] Determining a prediction loss of the auxiliary detection model based on the plurality of first pseudo-labels and the first sample label, wherein the prediction loss represents: an accuracy of the predicted pseudo-labels;

[0189] The object detection model is trained based on the obtained sampling consistency loss, transformation consistency loss and prediction loss.

[0190] In a possible embodiment, the processing module 802 is specifically configured to:

[0191] Adjust the model parameters of the auxiliary detection model based on the obtained sampling consistency loss, transformation consistency loss, and prediction loss;

[0192] Using the auxiliary detection model, predicting that the second sample image meets the third pseudo-label of the fine-grained type;

[0193] Using the object detection model, predict a first predicted label for the first sample image and predict a second predicted label for the second sample image;

[0194] Based on the obtained error between the first predicted label and the first sample label, and the error between the second predicted label and the third pseudo label, model parameters of the object detection model are adjusted.

[0195] In a possible embodiment, the processing module 802 is specifically configured to:

[0196] Determine the error between each two first pseudo-labels in the plurality of first pseudo-labels to obtain a sampling consistency error;

[0197] Based on the weighted average of the obtained sampling consistency errors, the sampling consistency loss of the auxiliary detection model is determined.

[0198] In a possible embodiment, the second pseudo-label is in the form of an image having the same image size as the corresponding transformed image; the processing module 802 is specifically configured to:

[0199] According to the image transformation strategy of the transformed image corresponding to one second pseudo-annotation, performing image transformation on the other second pseudo-annotations respectively to obtain corresponding transformed annotations;

[0200] Determine the error between each two annotations in each annotation including a second pseudo annotation and each transformed annotation obtained, and obtain each transformation consistency error;

[0201] Based on the weighted average of the obtained transformation consistency errors, the transformation consistency loss of the auxiliary detection model is determined.

[0202] In a possible embodiment, the processing module 802 is specifically configured to:

[0203] Determining prediction errors between the plurality of first pseudo-annotations and the first sample annotation respectively;

[0204] The prediction loss of the auxiliary detection model is determined based on the weighted average of the multiple prediction errors obtained.

[0205] In a possible embodiment, the processing module 802 is specifically configured to:

[0206] Based on the image flipping strategy, the second sample image is flipped to obtain a first transformed image;

[0207] Based on the image masking strategy, at least one specified covering pattern is used to mask the second sample image to obtain a second transformed image;

[0208] An auxiliary detection model is used to predict second pseudo-labels of the first transformed image and the second transformed image that meet the fine-grained type based on the first transformed image, the second transformed image, and the second sample label.

[0209] In a possible embodiment, the processing module 802 is further configured to:

[0210] After performing multiple rounds of iterative training based on the first sample image set and the second sample image set and outputting a trained object detection model, an image to be detected is obtained;

[0211] Use the target detection model to extract features from the image to be detected and obtain image features;

[0212] When a target detection model is used to determine that a specified target is contained in an image to be detected based on image features, a fine-grained type of prediction annotation output by the trained target detection model is obtained, wherein the prediction annotation is used to mark the position of the specified target in the image to be detected.

[0213] Based on the same inventive concept, the present invention provides a target detection device that can implement the functions corresponding to the aforementioned target detection method. Figure 9 , the device includes an acquisition module 901 and a processing module 902, wherein:

[0214] Acquisition module 901: used to acquire the image to be detected;

[0215] Processing module 902: used to use the target detection model to extract features from the image to be detected to obtain image features;

[0216] The processing module 902 is also used to use the target detection model to determine, based on image features, that the image to be detected contains a specified target, and obtain the fine-grained type prediction annotation output by the trained target detection model, wherein the fine-grained type characterization prediction annotation has an annotation accuracy reaching a preset threshold, and the prediction annotation is used to mark the position of the specified target in the image to be detected.

[0217] Please refer to Figure 10 The apparatus for training a target detection model can be run on a computer device 1000. The current version and historical versions of the data storage program and the application software corresponding to the data storage program can be installed on the computer device 1000. The computer device 1000 includes a processor 1080 and a memory 1020. In some embodiments, the computer device 1000 may include a display unit 1040, and the display unit 1040 includes a display panel 1041 for displaying a user interactive operation interface, etc.

[0218] In a possible embodiment, the display panel 1041 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0219] The processor 1080 is configured to read a computer program and then execute the method defined by the computer program. For example, the processor 1080 reads a data storage program or file, thereby running the data storage program on the computer device 1000 and displaying a corresponding interface on the display unit 1040. The processor 1080 may include one or more general-purpose processors and may also include one or more DSPs (Digital Signal Processors) to perform related operations to implement the technical solutions provided in the embodiments of the present application.

[0220] The memory 1020 generally includes internal memory and external memory, and the internal memory can be a random access memory (RAM), a read-only memory (ROM), and a cache (CACHE) etc. The external memory can be a hard disk, an optical disk, a USB disk, a floppy disk or a tape drive etc. The memory 1020 is used to store computer programs and other data. The computer program includes the application corresponding to each client etc., and other data may include data generated after the operating system or application is run, and the data includes system data (such as configuration parameters of the operating system) and user data. In the embodiment of the present application, program instructions are stored in the memory 1020, and the processor 1080 executes the program instructions in the memory 1020 to implement any of the methods discussed in the previous figure.

[0221] The display unit 1040 is used to receive input digital information, character information, or contact touch operations / contactless gestures, and to generate signal input related to user settings and function control of the computer device 1000. Specifically, in the embodiment of the present application, the display unit 1040 may include a display panel 1041. The display panel 1041, such as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any other suitable object or accessory on or on the display panel 1041) and drive corresponding connected devices according to a pre-set program.

[0222] In one possible embodiment, the display panel 1041 may include a touch detection device and a touch controller. The touch detection device detects the player's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller. The touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 1080. The touch controller can also receive and execute commands from the processor 1080.

[0223] The display panel 1041 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the display unit 1040, in some embodiments, the computer device 1000 may further include an input unit 1030. The input unit 1030 may include an image input device 1031 and other input devices 1032. The other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, a joystick, and the like.

[0224] In addition to the above, the computer device 1000 may also include a power supply 1090 for powering other modules, an audio circuit 1060, a near-field communication module 1070, and an RF circuit 1010. The computer device 1000 may also include one or more sensors 1050, such as an accelerometer, a light sensor, a pressure sensor, etc. The audio circuit 1060 specifically includes a speaker 1061 and a microphone 1062. For example, the computer device 1000 can use the microphone 1062 to collect the user's voice and perform corresponding operations.

[0225] As an embodiment, the number of the processors 1080 may be one or more, and the processor 1080 and the memory 1020 may be coupled or relatively independently configured.

[0226] As an example, Figure 10 The processor 1080 in the embodiment can be used to implement the following Figure 8 The functions of the acquisition module 801 and the processing module 802 in Figure 9 The functions of the acquisition module 901 and the processing module 902 in .

[0227] As an example, Figure 10 The processor 1080 can be used to implement the corresponding functions of the server or terminal device discussed above.

[0228] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0229] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the existing technology, can be embodied in the form of a software product, for example, through a computer program product, which is stored in a storage medium and includes a number of instructions for enabling a computer device to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0230] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

[0231] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for training a target detection model, characterized in that: include: Acquire a first sample image set and a second sample image set, wherein each first sample image is associated with a first sample annotation of a fine-grained type, and each second sample image is associated with a second sample annotation of a coarse-grained type, and the annotation accuracy of the fine-grained type is higher than that of the coarse-grained type; Based on the first sample image set and the second sample image set, multiple rounds of iterative training are performed to output a trained object detection model, wherein each round of iteration includes: Sampling the first sample annotation of the first sample image to obtain a plurality of sample annotations that conform to the coarse-grained type, and using the auxiliary detection model to predict first pseudo annotations that conform to the fine-grained type corresponding to each of the plurality of sample annotations; Using the auxiliary detection model, based on multiple transformed images corresponding to the second sample image and the corresponding second sample annotations, respectively predict second pseudo annotations that correspond to each of the multiple transformed images and conform to the fine-grained type; The target detection model is trained based on the obtained multiple first pseudo-annotations and multiple second pseudo-annotations.

2. The method according to claim 1, characterized in that The training of the target detection model based on the obtained multiple first pseudo-annotations and multiple second pseudo-annotations includes: Determining a sampling consistency loss of the auxiliary detection model based on the obtained multiple first pseudo-annotations, wherein the sampling consistency loss represents: consistency of predictions based on the multiple sampling annotations; Determining a transformation consistency loss of the auxiliary detection model based on the obtained plurality of second pseudo-annotations, wherein the transformation consistency loss represents: consistency of predictions based on a plurality of transformed images; Determining a prediction loss of the auxiliary detection model based on the multiple first pseudo-labels and the first sample label, wherein the prediction loss represents: an accuracy of the predicted pseudo-labels; The object detection model is trained based on the obtained sampling consistency loss, transformation consistency loss and prediction loss.

3. The method according to claim 2, characterized in that The target detection model is trained based on the obtained sampling consistency loss, transformation consistency loss and prediction loss, including: Adjusting model parameters of the auxiliary detection model based on the obtained sampling consistency loss, transformation consistency loss, and prediction loss; Using the auxiliary detection model, predicting that the second sample image conforms to a third pseudo-annotation of the fine-grained type; Using the object detection model, predict a first predicted label for the first sample image, and predict a second predicted label for the second sample image; Based on the obtained error between the first predicted label and the first sample label, and the error between the second predicted label and the third pseudo label, the model parameters of the object detection model are adjusted.

4. The method according to claim 2, characterized in that The determining, based on the obtained plurality of first pseudo-annotations, the sampling consistency loss of the auxiliary detection model includes: respectively determining an error between every two first pseudo-labels in the plurality of first pseudo-labels to obtain a sampling consistency error; Based on the obtained weighted average of the sampling consistency errors, the sampling consistency loss of the auxiliary detection model is determined.

5. The method according to claim 2, characterized in that The second pseudo-annotation is in the form of an image having the same image size as the corresponding transformed image; Then, determining the transformation consistency loss of the auxiliary detection model based on the obtained multiple second pseudo-annotations includes: According to the image transformation strategy of the transformed image corresponding to one second pseudo-annotation, performing image transformation on the other second pseudo-annotations respectively to obtain corresponding transformed annotations; Determining the error between each two annotations in each annotation including the second pseudo annotation and each obtained transformed annotation to obtain each transformation consistency error; Based on the obtained weighted average of the transformation consistency errors, the transformation consistency loss of the auxiliary detection model is determined.

6. The method according to any one of claims 1 to 5, characterized in that After performing multiple rounds of iterative training based on the first sample image set and the second sample image set and outputting a trained object detection model, the method further includes: Obtain the image to be detected; Using the target detection model, extracting features from the image to be detected to obtain image features; When the target detection model is used to determine, based on the image features, that the image to be detected contains a specified target, a fine-grained type of prediction annotation output by the trained target detection model is obtained, wherein the prediction annotation is used to mark the position of the specified target in the image to be detected.

7. A device for training a target detection model, characterized in that: include: An acquisition module is configured to acquire a first sample image set and a second sample image set, wherein each first sample image is associated with a first sample annotation of a fine-grained type, and each second sample image is associated with a second sample annotation of a coarse-grained type, wherein the annotation accuracy of the fine-grained type is higher than that of the coarse-grained type; A processing module is configured to perform multiple rounds of iterative training based on the first sample image set and the second sample image set, and output a trained object detection model, wherein each round of iteration includes: The processing module is specifically configured to: perform sampling processing on the first sample annotation of the first sample image to obtain a plurality of sample annotations that conform to the coarse-grained type, and use the auxiliary detection model to predict first pseudo annotations that conform to the fine-grained type corresponding to each of the plurality of sample annotations; The processing module is further configured to: use the auxiliary detection model to predict, based on the plurality of transformed images corresponding to the second sample image and the corresponding second sample annotations, second pseudo annotations that correspond to the plurality of transformed images and conform to the fine-grained type; The processing module is further configured to train a target detection model based on the obtained multiple first pseudo labels and multiple second pseudo labels.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: include: a memory for storing program instructions; The processor is configured to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 6 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method according to any one of claims 1 to 6.

Images