Image detection model training, image detection method, device, equipment and medium

Abstract

The present disclosure provides an image detection model training, image detection method, device, equipment and medium, which relate to the field of artificial intelligence, specifically to the field of deep learning and image processing, etc. The specific implementation scheme is: obtaining training image data, the training image data includes annotated images and unannotated images; detecting the training image data to obtain detection results and corresponding confidences; dividing the unannotated images according to the detection results and corresponding confidences of the annotated images to obtain pseudo-annotated images and other images; based on the annotated images and the pseudo-annotated images, supervised training is performed on the image detection model, and based on the other images, unsupervised training is performed on the image detection model. The embodiments of the present disclosure can improve the model training speed.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, specifically to the field of deep learning and image processing, and in particular to an image detection model training, image detection method, device, equipment and medium. Background Art

[0002] Deep learning has been widely used in various fields of social production, such as finance, industry, and education.

[0003] The two main driving directions of deep learning are models and data. Model innovation is a long process, so to apply deep learning to specific business scenarios, data-driven is often the most important. Summary of the invention

[0004] The present disclosure provides an image detection model training, an image detection method, an apparatus, a device and a medium.

[0005] According to one aspect of the present disclosure, there is provided an image detection model training method, comprising:

[0006] Acquire training image data, wherein the training image data includes annotated images and unannotated images;

[0007] Performing detection on the training image data to obtain a detection result and a corresponding confidence level;

[0008] According to the detection result of the labeled image and the corresponding confidence level, the unlabeled image is divided to obtain pseudo-labeled images and other images;

[0009] Based on the labeled images and the pseudo-labeled images, supervised training is performed on the image detection model; and based on the other images, unsupervised training is performed on the image detection model.

[0010] According to another aspect of the present disclosure, there is provided an image detection method, comprising:

[0011] Get the target image;

[0012] The target image is input into an image detection model to obtain a detection result of the target image, and the image detection model is trained using the image detection model training method described in any embodiment of the present disclosure.

[0013] According to one aspect of the present disclosure, there is provided an image detection model training device, comprising:

[0014] A training image acquisition module, used to acquire training image data, wherein the training image data includes annotated images and unannotated images;

[0015] A detection confidence determination module, used to detect the training image data to obtain a detection result and a corresponding confidence;

[0016] An unlabeled image segmentation module is used to segment the unlabeled image according to the detection result of the labeled image and the corresponding confidence level to obtain pseudo-labeled images and other images;

[0017] A model training module is used to perform supervised training on the image detection model based on the labeled images and the pseudo-labeled images, and to perform unsupervised training on the image detection model based on the other images.

[0018] According to another aspect of the present disclosure, there is provided an image detection device, comprising:

[0019] A target image acquisition module, used for acquiring a target image;

[0020] The target image detection module is used to input the target image into an image detection model to obtain a detection result of the target image. The image detection model is trained using the image detection model training method described in any embodiment of the present disclosure.

[0021] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image detection model training method or the image detection method described in any embodiment of the present disclosure.

[0025] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the image detection model training method or the image detection method described in any embodiment of the present disclosure.

[0026] According to another aspect of the present disclosure, a computer program object is provided, including a computer program, which, when executed by a processor, implements the image detection model training method or the image detection method described in any embodiment of the present disclosure.

[0027] The disclosed embodiments can improve the model training speed.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0030] Figure 1 is a flowchart of an image detection model training method disclosed in an embodiment of the present disclosure;

[0031] Figure 2 is a flowchart of another image detection model training method disclosed in an embodiment of the present disclosure;

[0032] Figure 3 is a flowchart of another image detection model training method disclosed in an embodiment of the present disclosure;

[0033] Figure 4 is a flowchart of another image detection model training method disclosed in an embodiment of the present disclosure;

[0034] Figure 5 is a flowchart of another image detection model training method disclosed in an embodiment of the present disclosure;

[0035] Figure 6 is a flow chart of an image detection method disclosed in an embodiment of the present disclosure;

[0036] Figure 7 is a structural schematic diagram of an image detection model training device disclosed in an embodiment of the present disclosure;

[0037] Figure 8 is a structural schematic diagram of an image detection device disclosed in an embodiment of the present disclosure;

[0038] Fig. 9 It is a block diagram of an electronic device used to implement the image detection model training method or image detection method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0040] Figure 1This is a flowchart of an image detection model training method disclosed in an embodiment of the present disclosure. This embodiment can be applied to the case of image detection model training. The method of this embodiment can be executed by an image detection model training device, which can be implemented in software and / or hardware, and is specifically configured in an electronic device with certain data computing capabilities. The electronic device can be a client device or a server device, and the client device can be a mobile phone, a tablet computer, a vehicle terminal, and a desktop computer.

[0041] S101. Acquire training image data, where the training image data includes annotated images and unannotated images.

[0042] The training image data includes a large number of training images. Annotated images refer to images with annotation information. Unannotated images refer to images without annotation information. Annotation information is used for supervised training models. Specifically, the annotation information can be classification information, recognition information, processing results, and degree values. The training image data can replace at least one of the following: text, image, voice or video. Exemplarily, the image detection model is used to classify images, and the annotation information can be the classification result. For example, the image detection model is used to identify targets in images, and the annotation information can be the target box corresponding to the target object and the type of the target object. For example, the image detection model is used to identify the clarity of images, and the annotation information can be the clarity level. In addition, other scenarios can also be used, which are not specifically limited.

[0043] S102: Detect the training image data to obtain a detection result and a corresponding confidence level.

[0044] Detecting the training image data means detecting each training image in the training image data, and obtaining a detection result and a confidence level corresponding to the detection result for each training image. The detection result is the result output by the image detection model. The confidence level corresponding to the detection result is the reliability of the detection result. Among them, the detection may include at least one of the following: classification, recognition, processing and degree detection, etc. Exemplarily, the image detection model is used to perform target detection in an image, specifically to determine at least one target frame and the confidence level of each target frame in the image. Among them, the detection result is all target frames, and the confidence level corresponding to the detection result is the confidence level of all target frames.

[0045] Some reliable methods can be used to detect the training image data. For example, classification can use a clustering algorithm, or a pre-trained reliable model can be used for detection.

[0046] S103 . Divide the unannotated image according to the detection result of the annotated image and the corresponding confidence level to obtain pseudo-annotated images and other images.

[0047] Unlabeled images include pseudo-labeled images and other images. Pseudo-labeled images are unlabeled images with reliable processing results. Other images are unlabeled images with unreliable processing results. Pseudo-labeled images combined with processing results can be used as annotation information for pseudo-labeled images, so that pseudo-labeled images can be transformed into labeled image training models. Other images are unlabeled images with unreliable processing results, and the detection results of other images cannot be used as annotation information for other images.

[0048] Reliable detection results can be screened out according to the confidence level, and the unlabeled images corresponding to the reliable detection results can be determined as pseudo-labeled images, and the unlabeled images corresponding to the unreliable detection results can be determined as other images. In addition, the pseudo-labeled images and other images can be further subdivided, or the unlabeled images can be provided with more images in addition to the pseudo-labeled images and other images. This can be set as needed, and there is no specific limitation on this.

[0049] S104: Perform supervised training on an image detection model based on the annotated image and the pseudo-annotated image, and perform unsupervised training on the image detection model based on the other images.

[0050] The detection result of the pseudo-annotated image is used as the annotation information of the pseudo-annotated image, and is used as an annotated image for supervised training. The annotated image is subjected to supervised training. The other images are subjected to unsupervised training. Optionally, the other images can train the model according to the features they include, for example, using the consistency features of other images, specifically the features that are still the same under different conditions in other images. The model can be trained by making the consistency features as similar as possible and making the inconsistent features as different as possible. Exemplarily, multiple algorithms or multiple methods are used to detect other images. Usually, the same image has the same extracted features or detection results for the same function, and different images have different extracted features or detection results for the same function. Accordingly, a loss function is constructed according to the foregoing content, and the model parameters are updated until the loss function converges, and the model training is completed.

[0051] According to the technical solution disclosed in the present invention, by screening unlabeled images through the prediction results and confidence levels of labeled images, the randomness of the division of pseudo-labeled images can be reduced, the accuracy and precision of the division of pseudo-labeled images can be improved, and different training methods can be performed for pseudo-labeled images and other images to achieve the utilization rate of training images, increase the diversity of training images, thereby increasing the representativeness of the content of training images, and improving the detection accuracy of the model. At the same time, the division process of unlabeled images can be simplified, the image division efficiency can be improved, and the model training speed can be increased.

[0052] Figure 2It is a flowchart of another image detection model training method disclosed in accordance with an embodiment of the present disclosure, which is further optimized and expanded based on the above technical solution, and can be combined with the above optional implementations. According to the detection results and corresponding confidences of the annotated images, the unannotated images are divided to obtain pseudo-annotated images and other images, which is specifically as follows: according to the detection results and corresponding confidences of the annotated images, the confidence threshold of at least one type of detection results is calculated; according to the confidence threshold of each type of detection result, the detection results and corresponding confidences of each unannotated image, each unannotated image is divided to obtain pseudo-annotated images and other images.

[0053] S201. Acquire training image data, where the training image data includes annotated images and unannotated images.

[0054] S202: Detect the training image data to obtain a detection result and a corresponding confidence level.

[0055] Optionally, the training image data is detected to obtain a detection result and a corresponding confidence level, including: using a pre-trained target model to detect the training image data to obtain a detection result and a corresponding confidence level, and the target model is independent of the image detection model.

[0056] The input of the target model is the same as the input of the image detection model; the output of the target model is the same as the output of the image detection model. The target model is used to detect the training image to obtain the detection result of the training image and the confidence level corresponding to the detection result. For example, the image detection model is used to identify the vehicle in the image, and accordingly, the target model is also used to identify the vehicle in the image. For another example, the image detection model is used for image classification, and the target model is also used for image classification.

[0057] The target model is a reliable trained model. The model structure of the target model can be the same as or different from that of the image detection model. The training of the target model and the training of the image detection model are independent of each other. The target model can be understood as a complex, high-accuracy and high-precision model. The image detection model can be understood as a simple and lightweight model.

[0058] By training the image detection model based on the accurate detection results of the training images based on the target model, the image segmentation accuracy can be improved, thereby improving the detection accuracy of the image detection model. At the same time, the training images are detected based on the target model to improve the detection speed and the image segmentation speed.

[0059] S203: Calculate a confidence threshold of at least one type of detection result according to the detection result of the annotated image and the corresponding confidence level.

[0060] There are multiple types of detection results in the training image. The detection results can be divided to obtain at least one type of detection result, and for each type, the confidence is statistically calculated to obtain the confidence of each type of detection result. According to the confidence of each type, the distribution of the confidence of each type is determined, and according to the concentrated confidence under a type, the confidence threshold of the detection result of this type is determined. In fact, for each type, the distribution of the confidence can be used to measure the majority of the detection results of this type, thereby screening out reliable confidences with high frequency, and thus determining the confidence threshold to screen reliable detection results of unlabeled images, which can improve the screening accuracy.

[0061] Exemplarily, the confidence threshold can be determined by taking the mean or median of the confidences under a type; or the confidences under a type are counted and the confidence with the largest number is determined as the confidence threshold. Other situations are also possible and are not specifically limited to this.

[0062] It should be noted that if the detection result of the annotated image is different from the annotation information, it is determined that the annotated image is detected incorrectly, and the annotated image does not participate in the calculation of the confidence threshold.

[0063] S204 , dividing each of the unlabeled images according to the confidence threshold of each type of detection result, the detection result of each of the unlabeled images and the corresponding confidence, to obtain pseudo-labeled images and other images.

[0064] The confidence threshold is used to classify pseudo-annotated images and other images. For example, unannotated images whose confidence in the detection result is greater than or equal to the confidence threshold can be determined as pseudo-annotated images; unannotated images whose confidence in the detection result is less than the confidence threshold can be determined as other images.

[0065] In the existing method, the confidence threshold is set by artificial experience, which makes the confidence threshold subjective and random. By determining the confidence threshold by the confidence statistics of the labeled image, the stability and objectivity of the confidence threshold can be improved, thereby improving the accuracy of the confidence threshold.

[0066] Optionally, calculating the confidence threshold of at least one type of detection result based on the detection results of the annotated images and the corresponding confidence levels includes: classifying the detection results of each of the annotated images to obtain at least one type of detection result and the number of annotated images of each type; and calculating the confidence threshold of the detection result of the type based on the number of annotated images of the type and the confidence level corresponding to the detection result of the type.

[0067] The number of images is the number of correct detection results of a type, and also the number of annotated images with the correct detection results of the type. In fact, the annotation information of the annotated image can be understood as correct information. A correct detection result means that the detection result is the same as the annotation information of the annotated image. The annotated images with the same annotation information as the detection result can be first screened out, and the number of detection results of the same type can be screened out from them, and determined as the number of images of the type. According to the number of images and the confidence value under the type, the confidence mean is calculated and determined as the confidence threshold.

[0068] Exemplarily, for an image detection model for image classification, there is a type of detection result of category c, and the confidence threshold Tc of category c is calculated based on the following formula:

[0069]

[0070] Where N is the number of images with classification result c and label information c, and Pi is the confidence that image i belongs to category c.

[0071] For an unlabeled image, its classification result c and the corresponding confidence Pc are obtained. If Pc is greater than or equal to Tc, the unlabeled image is divided into subset 1, otherwise the unlabeled image is divided into subset 2. Subset 1 includes pseudo-labeled images, and subset 2 is other images.

[0072] By counting the number of annotated images of a certain type and the confidence of the distribution under that type, counting the confidence mean, and calculating the confidence threshold of that type, we can accurately determine the confidence threshold based on the confidence distribution and divide the unlabeled images accordingly, which can improve the accuracy of unlabeled image division and the accuracy of unlabeled image division.

[0073] S205: Perform supervised training on the image detection model based on the annotated image and the pseudo-annotated image, and perform unsupervised training on the image detection model based on the other images.

[0074] Optionally, supervised training is performed on the image detection model based on the annotated images and the pseudo-annotated images, and unsupervised training is performed on the image detection model based on other images, including: calculating a supervised loss value based on the annotation information of the annotated images and the detection results of the pseudo-annotated images; calculating an unsupervised loss value based on the consistency features of other images; fusing the supervised loss value and the unsupervised loss value to obtain a total loss value; and training the image detection model based on the total loss value.

[0075] The supervised loss value and the unsupervised loss value are summed to obtain the total loss value. The image detection model is trained based on the total loss value. By distinguishing other images from pseudo-annotated images to calculate the loss, and performing supervised loss calculation on pseudo-annotated images and annotated images, the calculation method of different images is further subdivided on the loss, which can refine the impact of different images on the model, improve the accuracy of model detection, and take into account the application to all images, improve the utilization rate of images, reduce the cost of image generation and collection, and increase the speed of model training.

[0076] According to the technical solution disclosed in the present invention, by determining the confidence threshold through the confidence statistical information of the annotated image, the stability and objectivity of the confidence threshold can be improved, thereby improving the accuracy of the confidence threshold, accurately classifying unannotated images, and training the model in a targeted manner. While taking into account the representativeness of the images, the number of images can be increased, the cost of acquiring images can be reduced, and the efficiency and accuracy of model training can be improved.

[0077] Figure 3 It is a flowchart of another image detection model training method disclosed in accordance with an embodiment of the present disclosure, which is further optimized and expanded based on the above technical solution, and can be combined with the above optional implementation modes. Based on the other images, the image detection model is unsupervisedly trained, which is specifically as follows: data enhancement is performed on the other images to obtain a data enhanced image, the data enhanced image includes a first image and a second image, and the data enhancement method of the first image and the data enhancement method of the second image are different; for the same other image, the intra-image similarity between the features of the corresponding first image and the features of the corresponding second image is calculated; for different other images, the extra-image similarity between the features of the data enhanced images between two other images is calculated; based on the intra-image similarity and the extra-image similarity, an unsupervised loss value is calculated; and the image detection model is trained based on the unsupervised loss value.

[0078] S301: Acquire training image data, where the training image data includes annotated images and unannotated images.

[0079] S302: Detect the training image data to obtain a detection result and a corresponding confidence level.

[0080] S303: Divide the unannotated image according to the detection result of the annotated image and the corresponding confidence level to obtain pseudo-annotated images and other images.

[0081] S304: Perform supervised training on an image detection model based on the annotated image and the pseudo-annotated image.

[0082] Optionally, supervised training is performed on the image detection model based on the annotated image and the pseudo-annotated image, including: calculating a supervised loss value based on the annotation information of the annotated image and the detection result of the pseudo-annotated image.

[0083] The pseudo-annotated image and the annotated image can use the same method to calculate the loss value, or they can use different methods to calculate the loss value. The loss value of the pseudo-annotated image is fused with the loss value of the annotated image to obtain a supervised loss value. Specifically, the supervised loss value can be determined by summing the loss value of the pseudo-annotated image and the loss value of the annotated image.

[0084] For example, for labeled images, cross entropy loss can be used directly for supervision, and the calculated loss value is L1; for pseudo-labeled images, the detection results are directly used as labeling information, and the cross entropy loss is calculated for supervised training, and the loss value is L2, then the supervised loss value is L1+L2.

[0085] Optionally, weights may be set for the loss values ​​of different images, and the weighted sum of the loss value of the pseudo-annotated image and the loss value of the annotated image may be determined as the supervised loss value, and the supervised loss value is λ1*L1+λ2*L2.

[0086] S305. Perform data enhancement on the other images to obtain a data enhanced image, where the data enhanced image includes a first image and a second image, and the data enhancement method of the first image and the data enhancement method of the second image are different.

[0087] Data enhancement is used to increase the image's own features and to clearly differentiate it from other image features. The number of data-enhanced images is at least two, specifically including a first image and a second image. In addition, the data-enhanced images may also include a third image, a fourth image, etc. The enhancement methods between different images are different. Exemplarily, the number of data-enhanced images is two, including a first image and a second image. The data enhancement method can be determined according to the application scenario. Exemplarily, for an image detection model that is an image classification model, the image enhancement method may include color, shape brightness or contrast, etc.

[0088] The loss values ​​of different other images can be calculated in the same way or in different ways. Similarly, different weights can be preset for different ways. The loss values ​​of other images in different ways are weighted with different weights to obtain the unsupervised loss value.

[0089] S306 . For the same other image, calculate the intra-image similarity between the features of the corresponding first image and the features of the corresponding second image.

[0090] Intra-image similarity refers to the similarity between corresponding images of the same image obtained by different data enhancement methods. Specifically, the data enhancement image includes a first image and a second image, and the intra-image similarity is the similarity between the first image and the second image.

[0091] S307 . For different other images, calculate the out-of-image similarity between the features of the data enhanced images between two other images.

[0092] The external image similarity refers to the similarity between different images. Specifically, the external image similarity between other image A and other image B includes at least one of the following: the similarity between the first image of other image A and the first image of other image B, the similarity between the first image of other image A and the second image of other image B, and the similarity between the second image of other image A and the first image of other image B.

[0093] S308 : Calculate an unsupervised loss value according to the intra-image similarity and the extra-image similarity.

[0094] S309: Training the image detection model according to the unsupervised loss value.

[0095] With the goal of maximizing the similarity within the image and minimizing the similarity outside the image, the unsupervised loss values ​​of other images are constructed. The similarity between images can be specifically characterized by the similarity between the features of the images. For example, f1 is the feature of the first image, f2 is the feature of the second image, and the similarity within the image S(f1, f2) is calculated based on the following formula:

[0096]

[0097] For each other image, the intra-image similarity can be maximized and the extra-image similarity can be minimized. Accordingly, the loss value of the other image is a fraction, the numerator is the intra-image similarity, and the denominator is the sum of the intra-image similarity of the other image, any data-enhanced image of the other image and the extra-image similarity of any data-enhanced image of the remaining other images, that is, maximizing the loss value, which can achieve the proportion of maximizing the image content similarity. In order to achieve the effect of convergence, a minus sign is added before the fraction. Accordingly, the unsupervised loss value L3 is calculated based on feature comparison, specifically based on the following formula:

[0098]

[0099] in and They represent the features of two data augmentation methods for image i. M is the number of images, i and j are different.

[0100] Supervised training and unsupervised training are combined to jointly train the image detection model. Specifically, the supervised loss value and the unsupervised loss value are fused to obtain the total loss value. The supervised loss value and the unsupervised loss value can be accumulated or weighted with corresponding weights.

[0101] As in the previous example, L = λ1*L1+λ2*L2+λ3*L3.

[0102] The parameters of the image detection model are updated according to the total loss value until the total loss value converges or is minimized, and the training is determined to be complete.

[0103] Optionally, the data enhancement method includes at least one of the following: image geometric transformation and image pixel transformation.

[0104] In the disclosed embodiment, the image detection model is an image classification model, and the data enhancement method is an image enhancement method. Image geometric transformation is used to adjust the whole. Image pixel transformation is used to adjust the pixel level. Exemplarily, image geometric transformation may include: cropping, rotation and translation, etc., and image pixel transformation includes: brightness, contrast and color, etc. In a specific example, the first image and the second image are obtained by image geometric transformation, but the first image is cropped and the second image is rotated. Optionally, the first image is obtained by image geometric transformation and the second image is obtained by image pixel transformation. Image geometric transformation and image pixel transformation are different types of enhancement methods, and using a completely independent enhancement method can further increase the difference between the first image and the second image, highlight the difference between the first image and the second image, thereby improving the representativeness between the first image and the second image, and training the model by the first image and the second image to improve the accuracy of model detection.

[0105] By setting the data augmentation method to image geometric transformation and image pixel transformation, when the detection results of other images are unavailable, the consistent content of other images is added to increase the availability of other images, thereby enhancing the model's feature extraction capability for unlabeled images.

[0106] According to the technical solution disclosed in the present invention, by using different data enhancement methods to compare and calculate the loss using the features in other images that are consistent in the data enhancement methods, usable content can be extracted from other images to calculate the loss, thereby improving the utilization rate of other images and enhancing the model's feature extraction capability for other images, thereby reducing invalid images, reducing the cost of acquiring images, and improving model training efficiency.

[0107] Figure 4 and Figure 5 The present invention provides a flowchart of an image detection model training method according to the technical solution of the present invention. In the application scenario of the image detection model for image classification.

[0108] In the first stage, as Figure 4 shown, on the one hand, the labeled images are input into the pre-trained target model Q to obtain the detection results of the labeled images and the corresponding confidence levels. The target model Q is for image classification. The detection results of each labeled image are classified to obtain at least one type of detection results and the number of labeled images under each type; according to the number of labeled images under the type and the confidence level corresponding to the detection result, the confidence level threshold of the detection result of the type is calculated. Specifically, for a certain category c, the confidence level threshold Tc of category c is calculated based on the following formula:

[0109]

[0110] where N is the number of images whose all classification results are c and the annotation information is c, and Pi is the confidence level that image i belongs to category c.

[0111] On the other hand, the unlabeled images are input into the pre-trained target model Q to obtain the detection results of the unlabeled images and the corresponding confidence levels.

[0112] If the detection result of the unlabeled image is category c and the corresponding confidence level is Pc, compare Pc and Tc. When Pc≥Tc, the unlabeled image is divided into subset 1 as pseudo-labeled images; when Pc<Tc, the unlabeled image is divided into subset 2 as other images.

[0113] After the unlabeled images are divided, as Figure 5 shown, the image detection model F is trained based on the divided images.

[0114] All images are input into the image detection model F. For the labeled images and subset 1, direct supervision is carried out using the cross-entropy loss. The cross-entropy loss value of the labeled images is L1, and subset 1 is considered as reliable annotation information for the detection results, and the cross-entropy loss value of subset 1 is L2.

[0115] The unsupervised loss value L3 is calculated based on feature comparison. 1. Assume that a certain batch contains M other images in subset 2, then two different data augmentation methods are adopted for these M other images (for example, method 1 is cropping, rotating or translating, etc., and method 2 is changing brightness or contrast) to obtain 2*M other images; 2. Input the 2*M other images into the image detection model F to calculate 2*M features; 3. Calculate the cosine similarity between a single feature and the remaining (2*M - 1) features. The calculation formula is as follows:

[0116]

[0117] 4. Calculate the loss based on feature comparison:

[0118]

[0119] The three losses are fused through the proportional coefficient, with a supervised loss value of λ1*L1+λ2*L2, and the total loss value L=λ1*L1+λ2*L2+λ3*L3 is calculated. The loss ratio can be adjusted according to the actual training situation, and the sum of the ratios must be 1. The image detection model F is supervised and trained using the total loss value. According to the technical solution disclosed in the present invention, a semi-supervised training scheme based on pseudo-labeling and feature comparison is proposed, which aims to save the labeling time and cost of business data, and optimizes the model training with a small number of labeled images and a large number of unlabeled images; compared with existing methods, the threshold of pseudo-labeling for each category can be obtained by calculation, without the need for setting, and full use is made of unlabeled images.

[0120] Figure 6 This is a flowchart of an image detection method disclosed in an embodiment of the present disclosure. This embodiment can be applied to the application of image detection models. The method of this embodiment can be executed by an image detection device, which can be implemented in software and / or hardware, and is specifically configured in an electronic device with certain data computing capabilities. The electronic device can be a client device or a server device, and the client device can be a mobile phone, a tablet computer, a vehicle terminal, and a desktop computer.

[0121] S601: Acquire a target image.

[0122] The target image is used as the input of the image detection model. The target image can be a real-time image acquired, or a pre-stored local image. The method for acquiring the target image can be set according to the actual situation and is not specifically limited.

[0123] S602, inputting the target image into an image detection model to obtain a detection result of the target image, wherein the image detection model is trained using the image detection model training method described in any embodiment of the present disclosure.

[0124] The image detection model is used to detect the target image and obtain the detection result of the target image. Exemplarily, the image detection model is used to classify the image, specifically to detect the type of the target image. Exemplarily, the target image is an image including a vehicle, and the image detection model is used to determine the type of the vehicle in the target image. For another example, the target image is an image of a bill, and the image detection model is used to determine the type of the bill in the target image. For another example, the target image is an image of an industrial processed product, and the image detection model is used to determine whether there are defects in the product in the target image.

[0125] According to the technical solution of the present disclosure, an image detection model is obtained through the image detection model training method provided by the present disclosure, and image detection is performed on the target image based on the image detection model. This can quickly realize the image detection function, improve the detection efficiency of image detection, reduce the labor cost of image detection, and at the same time take into account the accuracy of image detection.

[0126] According to an embodiment of the present disclosure, Figure 7 1 is a structural diagram of an image detection model training device in an embodiment of the present disclosure, and the embodiment of the present disclosure is applicable to the case of model training. The device is implemented by software and / or hardware, and is specifically configured in an electronic device with certain data computing capabilities.

[0127] like Figure 7 The image detection model training device 700 shown in the figure includes: a training image acquisition module 701, a detection confidence determination module 702, an unlabeled image segmentation module 703 and a model training module 704.

[0128] A training image acquisition module 701 is used to acquire training image data, wherein the training image data includes annotated images and unannotated images;

[0129] A detection confidence determination module 702 is used to detect the training image data to obtain a detection result and a corresponding confidence;

[0130] The unlabeled image segmentation module 703 is used to segment the unlabeled image according to the detection result of the labeled image and the corresponding confidence level to obtain pseudo-labeled images and other images;

[0131] The model training module 704 is used to perform supervised training on the image detection model based on the labeled images and the pseudo-labeled images, and to perform unsupervised training on the image detection model based on the other images.

[0132] According to the technical solution disclosed in the present invention, by dividing the unlabeled images according to the prediction results and confidence levels of the labeled images, the randomness of the division of the pseudo-labeled images can be reduced, the accuracy and precision of the division of the pseudo-labeled images can be improved, and different training methods can be performed for the pseudo-labeled images and other images to achieve the utilization rate of the training images, increase the diversity of the training images, thereby increasing the representativeness of the content of the training images, and improving the detection accuracy of the model. At the same time, the division process of the unlabeled images can be simplified, the image division efficiency can be improved, and the model training speed can be increased.

[0133] Furthermore, the unlabeled image segmentation module 703 includes: a confidence threshold calculation unit, used to calculate the confidence threshold of at least one type of detection result according to the detection result of the labeled image and the corresponding confidence; an unlabeled image segmentation unit, used to divide each of the unlabeled images according to the confidence threshold of each type of detection result, the detection result of each of the unlabeled images and the corresponding confidence, to obtain pseudo-labeled images and other images.

[0134] Furthermore, the confidence threshold calculation unit includes: a detection result statistics subunit, which is used to classify the detection results of each of the annotated images, obtain at least one type of detection result, and the number of annotated images under each type; a confidence threshold detection subunit, which is used to calculate the confidence threshold of the detection result of the type according to the number of annotated images under the type and the confidence corresponding to the detection result of the type.

[0135] Furthermore, the model training module 704 includes: a data enhancement unit, which is used to perform data enhancement on the other images to obtain a data enhanced image, wherein the data enhanced image includes a first image and a second image, and the data enhancement method of the first image and the data enhancement method of the second image are different; an internal similarity calculation unit, which is used to calculate the internal image similarity between the features of the corresponding first image and the features of the corresponding second image for the same other image; an external similarity calculation unit, which is used to calculate the external image similarity between the features of the data enhanced images between two other images for different other images; an unsupervised loss detection unit, which is used to calculate the unsupervised loss value according to the internal image similarity and the external image similarity; and an unsupervised training unit, which is used to train the image detection model according to the unsupervised loss value.

[0136] Furthermore, the data enhancement method includes at least one of the following: image geometric transformation and image pixel transformation.

[0137] Furthermore, the detection confidence determination module 702 includes: a pre-trained model detection unit, which is used to use a pre-trained target model to detect the training image data to obtain a detection result and a corresponding confidence, and the target model is independent of the image detection model.

[0138] The above-mentioned image detection model training device can execute the image detection model training method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the image detection model training method.

[0139] According to an embodiment of the present disclosure, Figure 81 is a structural diagram of an image detection device in an embodiment of the present disclosure, and the embodiment of the present disclosure is applicable to the application of image detection models. The device is implemented by software and / or hardware and is specifically configured in an electronic device with certain data computing capabilities.

[0140] like Figure 8 The image detection device 800 shown in the figure includes: a target image acquisition module 801 and a target image detection module 802.

[0141] The target image acquisition module 801 is used to acquire the target image;

[0142] The target image detection module 802 is used to input the target image into an image detection model to obtain a detection result of the target image. The image detection model is trained using the image detection model training method described in any embodiment of the present disclosure.

[0143] According to the technical solution of the present disclosure, an image detection model is obtained through the image detection model training method provided by the present disclosure, and image detection is performed on the target image based on the image detection model. This can quickly realize the image detection function, improve the detection efficiency of image detection, reduce the labor cost of image detection, and at the same time take into account the accuracy of image detection.

[0144] The above-mentioned image detection device can execute the image detection method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the image detection method.

[0145] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0146] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program object.

[0147] Fig. 9 A schematic area diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0148] like Fig. 9As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0149] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0150] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as image detection model training methods or image detection methods. For example, in some embodiments, the image detection model training method or the image detection method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the image detection model training method or the image detection method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the image detection model training method or the image detection method in any other suitable manner (e.g., by means of firmware).

[0151] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard objects (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or the area diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0155] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0156] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services. The server may also be a server of a distributed system, or a server combined with a blockchain.

[0157] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, knowledge graph technology, and other major directions.

[0158] Cloud computing refers to a technology system that uses network access to elastically scalable shared physical or virtual resource pools. Resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for technical applications such as artificial intelligence and blockchain, as well as model training.

[0159] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution provided by this disclosure can be achieved, and this document does not limit this.

[0160] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for training an image detection model, comprising: Acquire training image data, wherein the training image data includes annotated images and unannotated images; Performing image detection on the training image data to obtain a detection result and a corresponding confidence level; Classifying the detection results of each of the labeled images to obtain at least one type of detection result; Calculating a confidence mean of the detection results of the type according to the number of images with annotated images of the type and the confidence corresponding to the detection results of the type, and determining the confidence mean as a confidence threshold of the detection results of the type; According to the confidence threshold of the detection result of each type, the detection result of each unlabeled image and the corresponding confidence, each unlabeled image is divided to obtain pseudo-labeled images and other images; Based on the labeled images and the pseudo-labeled images, supervised training is performed on the image detection model; and based on the other images, unsupervised training is performed on the image detection model.

2. The method according to claim 1, wherein: The performing unsupervised training on the image detection model based on the other images comprises: Performing data enhancement on the other images to obtain a data enhanced image, wherein the data enhanced image includes a first image and a second image, and a data enhancement method of the first image is different from a data enhancement method of the second image; For the same other image, calculating the intra-image similarity between the features of the corresponding first image and the features of the corresponding second image; For different other images, calculating the out-of-image similarity between the features of the data augmented images between two other images; Calculating an unsupervised loss value according to the intra-image similarity and the extra-image similarity; The image detection model is trained according to the unsupervised loss value.

3. The method according to claim 2, wherein: The data enhancement method includes at least one of the following: image geometric transformation and image pixel transformation.

4. The method according to claim 1, wherein: The detecting the training image data to obtain the detection result and the corresponding confidence level includes: The training image data is detected using a pre-trained target model to obtain a detection result and a corresponding confidence level, wherein the target model is independent of the image detection model.

5. An image detection method, comprising: Get the target image; The target image is input into an image detection model to obtain a detection result of the target image, wherein the image detection model is trained using the image detection model training method according to any one of claims 1 to 4.

6. An image detection model training device, comprising: A training image acquisition module, used to acquire training image data, wherein the training image data includes annotated images and unannotated images; A detection confidence determination module, used to detect the training image data to obtain a detection result and a corresponding confidence; An unlabeled image segmentation module is used to segment the unlabeled image according to the detection result of the labeled image and the corresponding confidence level to obtain pseudo-labeled images and other images; An image model training module, configured to perform supervised training on an image detection model based on the annotated image and the pseudo-annotated image, and to perform unsupervised training on the image detection model based on the other images; Wherein, the unlabeled image segmentation module includes: a confidence threshold calculation unit, configured to calculate a confidence threshold of at least one type of detection result according to the detection result of the annotated image and the corresponding confidence; an unlabeled image division unit, configured to divide each of the unlabeled images according to a confidence threshold of each type of detection result, the detection result of each of the unlabeled images and the corresponding confidence, to obtain pseudo-labeled images and other images; Wherein, the confidence threshold calculation unit includes: a detection result statistics subunit, configured to classify the detection results of the annotated images, and obtain at least one type of detection result and the number of annotated images of each type; The confidence threshold detection subunit is used to calculate the confidence mean of the detection results of the type according to the number of images with labeled images under the type and the confidence corresponding to the detection results of the type, and determine the confidence mean as the confidence threshold of the detection results of the type.

7. The device according to claim 6, wherein: The image model training module comprises: a data enhancement unit, configured to perform data enhancement on the other images to obtain a data enhanced image, wherein the data enhanced image includes a first image and a second image, and a data enhancement method of the first image is different from a data enhancement method of the second image; an intra-image similarity calculation unit, used for calculating, for the same other image, an intra-image similarity between a feature of a corresponding first image and a feature of a corresponding second image; An external similarity calculation unit, used for calculating, for different other images, an image external similarity between features of the data augmented image between two other images; An unsupervised loss detection unit, used for calculating an unsupervised loss value according to the intra-image similarity and the extra-image similarity; An unsupervised training unit is used to train the image detection model according to the unsupervised loss value.

8. The device according to claim 7, wherein: The data enhancement method includes at least one of the following: image geometric transformation and image pixel transformation.

9. The device according to claim 6, wherein: The detection confidence determination module comprises: The pre-trained model detection unit is used to detect the training image data using a pre-trained target model to obtain a detection result and a corresponding confidence level, wherein the target model is independent of the image detection model.

10. An image detection device, comprising: A target image acquisition module, used for acquiring a target image; A target image detection module is used to input the target image into an image detection model to obtain a detection result of the target image, wherein the image detection model is trained using the image detection model training method according to any one of claims 1 to 4.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image detection model training method described in any one of claims 1-4, or execute the image detection method described in claim 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the image detection model training method according to any one of claims 1 to 4, or to execute the image detection method according to claim 5.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the image detection model training method according to any one of claims 1 to 4, or executes the image detection method according to claim 5.

Images