Image processing method and apparatus

Abstract

Embodiments of the present disclosure provide an image processing method and device, the method comprising: training an image processing model through a first image set to obtain an image processing model after Nth training, N being greater than or equal to 1, sample data in the first image set being labeled data; filtering candidate sample data meeting a preset condition from a second image set through the image processing model after the Nth training, sample data in the second image set being unlabeled data; updating the first image set through the candidate sample data; and training the image processing model after the Nth training according to the updated first image set to obtain an image processing model after (N+1)th training. Thus, the relationship between the model and the data is complementary, the image processing model is used to enrich the training images, the model is trained based on the enriched training images, and the problem of poor model performance caused by insufficient images is solved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to an image processing method and apparatus. Background Technology

[0002] With the development of deep learning technology, deep learning models have broad application prospects in various data processing scenarios, such as image segmentation, image recognition, and image reconstruction.

[0003] Typically, during the training of a deep learning model, a large number of images are input into the deep learning model to obtain the prediction results. Based on the loss function, the loss value between the prediction results of the deep learning model and the true labels of the images is determined. The model parameters of the deep learning model are continuously updated based on the loss value to improve the image processing accuracy of the deep learning model.

[0004] The more images used and the more scenarios covered during the training of a deep learning model, the better the model's robustness. However, some deep learning tasks are difficult and costly to acquire images for training, making it hard to obtain a large number of training images, which leads to poor performance of the deep learning model. Summary of the Invention

[0005] This disclosure provides an image processing method and apparatus to overcome the problem of poor performance of deep learning models due to the difficulty in obtaining a large number of training images.

[0006] In a first aspect, embodiments of this disclosure provide an image processing model training method, including:

[0007] The image processing model is trained using the first image set to obtain the image processing model after the Nth round of training, where N is greater than or equal to 1, and the sample data in the first image set is labeled data.

[0008] The image processing model trained in the Nth round selects candidate sample data that meets the preset conditions from the second image set. The sample data in the second image set is unlabeled data.

[0009] Update the first image set using candidate sample data;

[0010] Based on the updated first image set, the image processing model trained in the Nth round is trained to obtain the image processing model trained in the N+1th round.

[0011] Secondly, embodiments of this disclosure provide an image processing method, including:

[0012] Acquire the target image;

[0013] The image processing model is used to predict whether the target to be detected exists in the target image, and the prediction result is obtained. The image processing model is trained according to the image processing model training method provided in the first aspect.

[0014] Thirdly, embodiments of this disclosure provide an image processing model training device, comprising:

[0015] The training unit is used to train the image processing model using the first image set to obtain the image processing model after the Nth round of training, where N is greater than or equal to 1, and the sample data in the first image set is labeled data.

[0016] The filtering unit is used to filter candidate sample data that meet preset conditions from the second image set through the image processing model trained in the Nth round. The sample data in the second image set is unlabeled data.

[0017] An update unit is used to update the first image set using candidate sample data;

[0018] The training unit is also used to train the image processing model after the Nth round of training based on the updated first image set, so as to obtain the image processing model after the N+1th round of training.

[0019] Fourthly, embodiments of this disclosure provide a data processing apparatus, including:

[0020] The acquisition unit is used to acquire the target image;

[0021] The prediction unit is used to predict whether the target to be detected exists in the target image through the image processing model and obtain the prediction result. The image processing model is trained by the image processing model training method provided in the first aspect.

[0022] Fifthly, embodiments of this disclosure provide an electronic device, including: at least one processor and a memory;

[0023] The memory stores the instructions that the computer executes;

[0024] At least one processor executes computer execution instructions stored in memory, causing at least one processor to execute the image processing model training method provided in the first aspect above, or to execute the image processing method provided in the second aspect.

[0025] In a sixth aspect, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the image processing model training method provided in the first aspect above, or implements the image processing method provided in the second aspect.

[0026] In a seventh aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, the computer program product including computer execution instructions, which, when executed by a processor, implement the image processing model training method provided in the first aspect above, or implement the image processing method provided in the second aspect.

[0027] The image processing method and apparatus provided in this embodiment train an image processing model using a first image set to obtain an image processing model after the Nth round of training. Using this Nth round of training, candidate sample data that meets certain conditions is selected from a second image set, and the first image set is updated based on the candidate sample data. Thus, by selecting from the second image set using the image processing model, the first image set is enriched. Subsequently, based on the updated first image set, the image processing model after the Nth round of training is trained again to obtain an image processing model after the (N+1)th round of training. Because the updated first image set is more comprehensive, the model accuracy of the image processing model after the (N+1)th round of training is higher than that of the image processing model after the Nth round of training.

[0028] As can be seen, in this embodiment, the complementary relationship between the model and the data is utilized (i.e., the model can filter out suitable data from the dataset, and the data can improve the model's accuracy). The image processing model enriches the labeled first image set used as training images by filtering the unlabeled second image set. This enriched first image set improves the accuracy of the image processing model. Therefore, the problem of insufficient training images leading to low model accuracy is solved. Users do not need to manually add labels to the unlabeled data, which increases the amount of training images, enhances their complexity and diversity, and improves the performance of the image processing model. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 An example diagram illustrating an application scenario provided by an embodiment of this disclosure;

[0031] Figure 2 Schematic diagram of the image processing model training method provided in the embodiments of this disclosure Figure 1 ;

[0032] Figure 3 Schematic diagram of the image processing model training method provided in the embodiments of this disclosure Figure 2 ;

[0033] Figure 4 This is a schematic flowchart of the image processing method provided in the embodiments of this disclosure;

[0034] Figure 5 This is a structural block diagram of the image processing model training device provided in the embodiments of this disclosure;

[0035] Figure 6 This is a structural block diagram of an image processing apparatus provided in an embodiment of the present disclosure;

[0036] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0038] In the training process of deep learning models, a large amount of data with real labels is input into the deep learning model. The deep learning model then performs target prediction on this data, and its parameters are continuously updated based on the loss function between the prediction result and the real label. For example, images labeled with the presence or absence of hand regions are input into a deep learning model. The model predicts whether hand regions exist in these images, and its parameters are continuously updated based on the loss function between the predicted presence and the actual presence or absence of hand regions. Generally, the larger the amount of data used to train the deep learning model, the more data scenarios it can cover, the more robust the deep learning model will be, and the higher its prediction accuracy.

[0039] However, in some data scenarios (such as medical image segmentation / recognition, image segmentation / recognition in hazardous scenes, and light field image recognition / segmentation), it is difficult to collect training images, or the cost of obtaining data labels for training images is too high, resulting in insufficient training image data. When the amount of training image data is insufficient, the accuracy and robustness of deep learning models will decrease, and for scenarios not covered by training images, deep learning models cannot perform accurate image processing.

[0040] To address the aforementioned problems, this disclosure proposes an image processing method that can improve model performance and image processing accuracy even with insufficient training data. In this embodiment, leveraging the synergistic relationship between the model and data, an image processing model is trained based on a first set of labeled images. The trained image processing model then filters a second set of unlabeled images, enriching the training image set with eligible images. The enriched training image set is then used to train the image processing model again. This further improves the performance of the image processing model. The performance of the image processing model includes both accuracy and robustness.

[0041] As can be seen, the embodiments of this disclosure do not require manual annotation of a large number of images, and can continuously enrich the training images, increase the complexity and diversity of the training images, save the high cost of collecting or annotating training images, and train a high-performance image processing model using only a small number of images.

[0042] refer to Figure 1 , Figure 1 This is an example diagram illustrating an application scenario provided by an embodiment of this disclosure.

[0043] like Figure 1 As shown, the embodiments of this disclosure are applicable to image processing model training scenarios. This scenario includes an image processing model training device 100 and a preset storage space 101. The image processing model training device 100 is equipped with an image processing model, and the preset storage space 101 stores a first set of labeled images and a second set of unlabeled images. During the training process of the image processing model, the image processing model is trained using the first image set. The image processing model then selects suitable images from the second image set to update the first image set, enriching it. Based on the updated first image set, the image processing model is trained again.

[0044] The image processing model training device 100 can be any electronic device configured with a processor and memory, or a collection of electronic devices, such as a server. The preset storage space 104 can be any storage space with data storage function, such as the local storage space of the model training device 100, the remote storage space on the server, etc., and this embodiment does not impose specific limitations.

[0045] The image processing model is a deep learning model.

[0046] The embodiments disclosed herein are applicable to various deep learning model training scenarios, especially to deep learning models (such as image segmentation models / image classification models / image recognition models) for various deep learning vision-oriented tasks.

[0047] refer to Figure 2 , Figure 2 A flowchart illustrating the image processing model training method provided in this embodiment. Figure 1 .like Figure 2 As shown, the training method for this image processing model includes:

[0048] S201, The image processing model is trained using the first image set to obtain the image processing model after the Nth round of training, where N is greater than or equal to 1, and the sample data in the first image set is labeled data.

[0049] The first image set includes multiple sample data (also known as sample images), each labeled. The labels describe whether the target to be detected actually appears in the sample data, and the image processing model can predict whether the target to be detected exists in the sample data. The first image set may include sample data containing the target to be detected (positive sample data) and sample data not containing the target to be detected (negative sample data) to improve the performance of the image processing model.

[0050] Where N is 1, the image processing model is an untrained image processing model. If N is greater than 1, the image processing model is the image processing model trained in the (N-1)th round.

[0051] In this embodiment, an image processing model can be used to perform target detection on sample data in the first image set to obtain a prediction result of whether the target to be detected exists in the sample data. The loss function of the image processing model is used to determine the loss value between the prediction result and the label of the sample data. Based on the loss value, the model parameters of the image processing model are adjusted to achieve the Nth round of training. In the Nth round of training, the image processing model can undergo one or more parameter adjustments. For example, the sample data in the first image set can be divided into batches, and the parameters of the image processing model can be adjusted once based on each batch of sample data. Here, there are no restrictions on the specific model training operations such as parameter adjustment and loss function calculation.

[0052] S202, using the image processing model trained in the Nth round, selects candidate sample data that meet the preset conditions from the second image set. The sample data in the second image set is unlabeled data.

[0053] The second image set is different from the first image set. The second image set may include sample data containing the target to be detected and sample data where the target to be detected does not exist.

[0054] In this embodiment, since the sample data in the second image set is unlabeled, it is unclear which sample data in the second image set contains the target to be detected and which sample data does not. However, after the Nth round of training, the image processing model possesses a certain target detection capability. Therefore, the image processing model can be used to perform target detection on the sample data in the second image set to obtain a prediction result of whether the sample data in the second image set contains the target to be detected. Then, based on the prediction result of the sample data in the second image set, sample data that meets preset conditions can be selected from the second image set. For simplicity, the sample data that meets the preset conditions is referred to as candidate sample data.

[0055] In one possible implementation, the preset condition is that the target to be detected exists in the sample data. In this case, S202 includes: performing target detection on the second image set using the image processing model trained in the Nth round to obtain sample data in the second image set that contains the target to be detected; and determining the candidate sample data as the sample data in the second image set that contains the target to be detected.

[0056] In this implementation, the image processing model, trained for the Nth round, performs target detection on the second image set to obtain a prediction result indicating whether the sample data in the second image set contains the target to be detected. Sample data from the second image set whose prediction result indicates the presence of the target to be detected are selected as candidate sample data. Thus, by utilizing the target detection capability of the image processing model, sample data containing the target to be detected is selected, enabling the labeling of unlabeled sample data without manual annotation. Subsequently, the selected sample data containing the target to be detected can be used to enrich the first image set, increasing the data volume of the first image set.

[0057] S203, Update the first image set using candidate sample data.

[0058] In this embodiment, after obtaining candidate sample data, the candidate sample data can be added to the first image set, or the candidate sample data can be further processed before being added to the first image set, resulting in an updated first image set. This increases the amount of data in the first image set and enhances its diversity. Since the prediction results of the candidate sample data obtained through the image processing model during the candidate sample data selection process are equivalent to the predicted labels of the candidate sample data, predicted labels can be added to the candidate sample data during the updating of the first image set. Then, the first image set is updated based on the candidate sample data with added predicted labels.

[0059] In one possible implementation, if the preset condition is that the target to be detected exists in the sample data—that is, if the candidate sample data consists of sample data containing the target to be detected selected from the second image set by the image processing model after the Nth round of training—the candidate sample data can be added to the first image set. Alternatively, the candidate sample data can be augmented (e.g., through geometric transformations such as rotation, cropping, and scaling, or through pixel transformations such as adding salt-and-pepper noise or Gaussian noise), and the augmented candidate sample data can be added to the first image set. This increases the number and diversity of sample data containing the target to be detected in the first image set, thereby improving the complexity and diversity of the first image set.

[0060] S204. Based on the updated first image set, train the image processing model after the Nth round of training to obtain the image processing model after the N+1th round of training.

[0061] In this embodiment, compared with the first image set before the update, the updated first image set has a larger data volume and more diverse data. Based on the updated first image set, the image processing model trained in the Nth round is trained in the N+1th round to obtain the image processing model trained in the N+1th round. The performance of the image processing model trained in the N+1th round is better than that of the image processing model trained in the Nth round, and the prediction accuracy of the image processing model trained in the N+1th round is higher than that of the image processing model trained in the Nth round.

[0062] The training methods for both the Nth and N+1th rounds of the image processing model are supervised training methods. However, the same or different optimization algorithms and loss functions can be used in the Nth and N+1th rounds. No restrictions are placed on the specific optimization algorithm and loss function used in the N+1th round of the image processing model.

[0063] Then, based on the image processing model trained in the N+1th round, candidate sample data that meets the preset conditions can be selected more accurately from the second image set. The first image set is updated using the candidate sample data, and the image processing model is trained in the N+2th round based on the updated first image set.

[0064] Thus, following the steps described above, the image processing model is trained in multiple rounds. During these multiple training rounds, as the number of training rounds increases, the complexity, diversity, and accuracy of the first image set increase, and the performance of the image processing model and the accuracy of object detection also improve.

[0065] In this embodiment, the second image set, as non-homogeneous data of the first image set, is used to increase the data volume, complexity, and diversity of the first image set under the filtering of the image processing model. The optimized first data model can further improve the performance and target detection accuracy of the image processing model. The optimized image processing model can then further refine the filtering of the second image set, adding more accurately labeled sample data to the first image set. Thus, by leveraging the synergistic relationship between the model and the data, the complexity, diversity, and accuracy of the sample data in the first image set are gradually increased, thereby gradually improving the performance and target detection accuracy of the image processing model, without the need for collecting a large number of images or manually labeling them.

[0066] In some embodiments, the second image set and the first image set are applied to different image processing tasks. For example, the first image set is used for image object recognition, while the second image set is used for other tasks such as image classification and image optimization (e.g., converting low-resolution images into high-resolution images). Thus, by using images from other task scenarios (i.e., the second image set) to enrich the training images (i.e., the first image set) of the image processing model, the training images cover more scenarios, increasing their complexity and diversity, and improving the robustness of the image processing model across various scenarios. In particular, through this embodiment, some sample data from the second image set can be transferred as prior knowledge for other tasks to the training process of the image processing model, resulting in faster convergence and higher accuracy in the training of the image processing model.

[0067] In some embodiments, the training of the image processing model is terminated in response to the number of training epochs of the image processing model being greater than an epoch threshold, or the screening accuracy of the image processing model being greater than an accuracy threshold.

[0068] In this embodiment, after the N+1th round of training of the image processing model, it can be determined whether N+1 is greater than a preset round number threshold. If so, the training of the image processing model ends; otherwise, the N+1th round of training of the image processing model continues.

[0069] Alternatively, after selecting candidate sample data that meets preset conditions from the second image set, the selection accuracy (i.e., selection precision) of the image processing model can be determined based on the difference between the candidate sample data selected by the image processing model after the Nth round of training and the candidate sample data selected in the (N-1)th round. If the selection precision of the image processing model is greater than the precision threshold, it indicates that the image processing model performs well and can accurately select candidate sample data from the second image set. Further training of the image processing model will not significantly improve the performance of the first image set, and the training of the image processing model can be terminated. Otherwise, the N+1th round of training of the image processing model continues.

[0070] Thus, by using the above training termination conditions, we can ensure that the image processing model has high performance after training and that the sample data in the first image set meets certain accuracy, complexity, and diversity requirements.

[0071] In addition to the aforementioned training termination conditions, training of the image processing model can also be terminated when the target detection accuracy of the image processing model on the first image set is greater than an accuracy threshold, thus ensuring the target detection accuracy of the image processing model. Specifically, the target detection accuracy of the image processing model on the first image set can be determined by comparing the image processing model's prediction of the presence of the target in the sample data of the first image set with the labels of the sample data in the first image set.

[0072] In some embodiments, the preset condition may be that the target to be detected does not exist in the sample data. Candidate sample data without the target to be detected can be selected from the second image set using the image processing model trained in the Nth round. See details... Figure 3 The example shown.

[0073] refer to Figure 3 , Figure 3 A flowchart illustrating the image processing model training method provided in this embodiment. Figure 2 .like Figure 3 As shown, the training method for this image processing model includes:

[0074] S301, The image processing model is trained using the first image set to obtain the image processing model after the Nth round of training, where N is greater than or equal to 1, and the sample data in the first image set is labeled data.

[0075] The implementation principle and technical effect of S301 can be referred to the aforementioned embodiments, and will not be repeated here.

[0076] S302, using the image processing model trained in the Nth round, target detection is performed on the second image set to obtain sample data in the second image set that do not contain the target to be detected, and the candidate sample data is determined to be sample data that does not contain the target to be detected.

[0077] In this embodiment, the image processing model trained in the Nth round performs target detection on the second image set to obtain a prediction result of whether the sample data in the second image set contains the target to be detected. Sample data from the second image set whose prediction result indicates the absence of the target to be detected are selected, and these are then identified as candidate sample data. Thus, by utilizing the target detection capability of the image processing model, sample data containing the target to be detected is selected without manual annotation, and the selected sample data containing the target to be detected can subsequently be used to enrich the first image set.

[0078] S303, Update the first image set using candidate sample data.

[0079] In this embodiment, new sample data can be added to the first image set based on the candidate sample data, and corresponding labels can be determined for the newly added sample data, thereby increasing the number of samples in the first image set and improving the complexity and diversity of the sample data in the first image set.

[0080] In one possible implementation, since the candidate sample data consists of sample data from the second image set that do not contain the target to be detected, filtered by the image processing model, the candidate sample data can be added to the first image set as negative sample data (i.e., sample data where the target to be detected does not exist). Alternatively, the candidate sample data can be further augmented, and then the augmented candidate sample data can be added to the first image set as negative sample data. In this way, an updated first image set is obtained, increasing the amount of data in the first image set and improving its diversity.

[0081] Considering that before the image processing model is fully trained, the candidate sample data obtained by the image processing model may also contain the target to be detected, but the image processing model just does not detect it, in order to improve the accuracy of the sample data added to the first image set, in another possible implementation, S303 includes: adding the target to be detected to the candidate sample data to obtain updated candidate sample data; and updating the first image set according to the updated candidate sample data.

[0082] In this implementation, a target to be detected is added to the candidate sample data. This transforms the candidate sample data from samples where the target is not predicted by the image processing model into samples where the target is actually present. After updating the candidate sample data, the updated candidate sample data is added as positive sample data to the first image set, thereby increasing the quantity, complexity, and diversity of positive sample data in the first image set. Compared to directly adding samples where the target is predicted by the image processing model to the first image set, or directly adding samples where the target is not predicted by the image processing model to the first image set, this implementation ensures the accuracy of the labels of the sample data added to the first image set. The higher the target detection accuracy of the image processing model, the closer the updated candidate sample data is to the actual sample data containing the target.

[0083] Furthermore, in this implementation scheme, different scenarios and different styles of targets to be detected can be added to the candidate sample data, thereby increasing the diversity and complexity of the targets to be detected in the candidate sample data, and thus increasing the diversity and complexity of the positive sample data in the updated first image set.

[0084] Optionally, the target to be detected is added to the candidate sample data to obtain updated candidate sample data. This includes: using the candidate sample data as a background image, synthesizing the sample data from a third image set with the candidate sample data to obtain updated candidate sample data, where the sample data from the third image set is a mask image of the target to be detected. Thus, through image synthesis, the candidate sample data is converted into sample data containing the actual target to be detected.

[0085] The process of combining the sample data from the third image set with the candidate sample data refers to pasting the mask image of the target to be detected from the third image set onto the candidate sample data.

[0086] In this optional method, for each candidate sample data, a sample data can be randomly selected from the third image set and pasted onto the candidate sample data to obtain updated candidate sample data.

[0087] Optionally, during the process of updating the first image set based on the updated candidate sample data: the updated candidate sample data is determined as the newly added sample data in the first image set; or, data augmentation processing is performed on the updated candidate sample data, and the data-augmented candidate sample data is determined as the newly added sample data in the first image set. Specifically, performing data augmentation processing on the updated candidate sample data can further increase the complexity and diversity of the newly added sample data in the first image set.

[0088] S304. Based on the updated first image set, train the image processing model after the Nth round of training to obtain the image processing model after the N+1th round of training.

[0089] The implementation principle and technical effect of S304 can be referred to the aforementioned embodiments, and will not be repeated here.

[0090] In this embodiment, the second image set, as non-originating data of the first image set, is sample data without the target to be detected selected by the image processing model in the second image set. This increases the data volume, complexity, and diversity of the first image set. The optimized first data model can further improve the performance and target detection accuracy of the image processing model. Furthermore, the optimized image processing model can select even purer sample data without the target to be detected from the second image set, adding more accurate sample data to the first image set. Thus, by utilizing the complementary relationship between the model and the data, the complexity, diversity, and accuracy of the sample data in the first image set are gradually increased, thereby gradually improving the performance and target detection accuracy of the image processing model, without the need to collect a large number of images or manually annotate them.

[0091] To further clarify the above scheme, we will use a "hand image segmentation task" as an example. In this case, the image processing model is a target segmentation model, used to segment the hand region appearing in the image. Assume that the first image set is image set A, and the images in image set A have real labels, which are used to represent the hand region segmented from the image; the second image set is image set B, and the images in image set B do not have labels.

[0092] Image set A is used to train the target segmentation model, and its data volume is relatively small; image set B is used to expand image set A. Image set B can be a public image set from other tasks (such as image recognition / classification tasks), such as ImageNet or Celeba image sets, so as to transfer prior knowledge from other tasks to the training of the target segmentation model in this embodiment, thereby improving the model convergence speed and model accuracy.

[0093] The training process of the object segmentation model based on image set A and image set B is as follows:

[0094] Step 1: The target segmentation model can be trained using image set A to obtain target segmentation model H_1.

[0095] Step 2: Use the target segmentation model H_1 to predict image set B, and obtain the hand region segmentation result for each image in image set B.

[0096] Step 3: Based on the hand region segmentation results of each image in image set B, filter image set B: If the hand region segmentation result of an image is 0, it means that the target segmentation model H1 believes that there is no "hand region" in this image, so such images can be filtered out as "clean images". In this way, the set of all "clean images" in image set B is obtained, namely "clean image set B_1".

[0097] Step 4: Combine the "clean image set B_1" with a pre-prepared hand mask image (which can come from the label in image set A or from other image sets) to create a new image set, called image set "B_composite_1". The specific composite process includes: using the images in "clean image set B_1" as background images, pasting the hand mask image onto this background image to obtain image set "B_composite_1".

[0098] Step 5: Train the target segmentation model H_1 using image set "B_synthesis_1" and image set A together to obtain the target segmentation model H_2. The target segmentation model H_2 is more accurate and robust than the target segmentation model H_1 because the training data of the target segmentation model H_2 covers more scenes, has richer backgrounds, and has a richer variety of styles and types of hand regions in the images.

[0099] Step 6: Use the target segmentation model H2 to predict image set B, and obtain the hand region segmentation result for each image in image set B.

[0100] Step 7: Based on the hand region segmentation results of each image in image set B, filter image set B: If the hand region segmentation result of an image is 0, it means that the target segmentation model H1 believes that there is no "hand region" in this image, so such images can be filtered out as "clean images". In this way, the set of all "clean images" in image set B is obtained, namely "clean image set B_2".

[0101] The "clean image set B_2" is "cleaner" and richer than the "clean image set B_1". Specifically, the "clean image set B_1" is the prediction result obtained by the target segmentation model H_1. On the one hand, for some complex image scenes, the target segmentation model H_1 cannot identify the hand region contained in the image, and mistakenly classifies images containing hand regions as "clean images". On the other hand, for some images that do not contain hand regions, the target segmentation model H_1 will mistakenly classify them as images containing hand regions, and will not select them as "clean images". However, the robustness and accuracy of the target segmentation model H_2 are better than those of the target segmentation model H_1. Naturally, the target segmentation model H_2 performs better than the target segmentation model H_1 in the above two aspects, and the "clean image set B_2" selected by the target segmentation model H_2 is "cleaner" and richer.

[0102] Step 8: Combine the "clean image set B_2" with a pre-prepared hand mask image (which can come from the label in image set A or from other image sets) to create a new image set, called image set "B_composite_2". The specific composite process includes: using the images in "clean image set B_2" as background images, pasting the hand mask image onto this background image to obtain image set "B_composite_2".

[0103] Step 9: Train the model using image set "B_synthesis_2" and image set A together to obtain object segmentation model H_3. Object segmentation model H_3 is more robust than "hand_seg_model_iter2" because the training data of object segmentation model H_3 covers more scenes, has richer backgrounds, and has a richer variety of hand regions in the images.

[0104] In this way, new image sets are continuously synthesized in a loop, constantly preparing richer and more accurate training data for the target segmentation model, and training new target segmentation models. During this cycle, the synthesized image sets become increasingly rich and accurate. Using accurate data, the resulting model becomes increasingly robust and accurate. Throughout the process, the complementary relationship between the model and the data is utilized, continuously optimizing the accuracy of data synthesis and model training, providing an effective solution for model training under conditions of insufficient data.

[0105] The application process of the image processing model in any of the foregoing embodiments is described below through examples.

[0106] It should be noted that the application process of the image processing model and the training process of the image processing model can be carried out on the same device or on different devices.

[0107] refer to Figure 4 , Figure 4 This is a schematic flowchart illustrating the image processing method provided in an embodiment of this disclosure. Figure 4 As shown, the image processing methods include:

[0108] S401. Obtain the target image.

[0109] S402. The image processing model is used to predict whether the target to be detected exists in the target image, and the prediction result is obtained. The image processing model is trained according to any of the foregoing embodiments.

[0110] In this embodiment, the target image can be input into an image processing model for target detection to obtain a prediction result of whether the target to be detected exists in the target image. Here, the target image is the image to be detected, and the target to be detected is, for example, a hand region. The target detection process of the image processing model is not limited, nor is the specific model structure of the image processing model limited.

[0111] The image processing model training method corresponding to the above embodiment is referred to... Figure 5 , Figure 5 This is a structural block diagram of an image processing model training device provided in an embodiment of this disclosure. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. Figure 5 As shown, the model training device 500 includes: a training unit 501, a screening unit 502, and an update unit 503.

[0112] Training unit 501 is used to train the image processing model using the first image set to obtain the image processing model after the Nth round of training, where N is greater than or equal to 1, and the sample data in the first image set is labeled data.

[0113] The filtering unit 502 is used to filter candidate sample data that meet preset conditions from the second image set through the image processing model trained in the Nth round. The sample data in the second image set is unlabeled data.

[0114] Update unit 503 is used to update the first image set using candidate sample data;

[0115] Training unit 501 is also used to train the image processing model after the Nth round of training based on the updated first image set, so as to obtain the image processing model after the N+1th round of training.

[0116] According to one or more embodiments of the present disclosure, the training unit 501 is further configured to: terminate the training of the image processing model in response to the number of training rounds of the image processing model being greater than a round number threshold or the screening accuracy of the image processing model being greater than an accuracy threshold.

[0117] According to one or more embodiments of this disclosure, the preset condition is that there is no target to be detected in the sample data. In the process of selecting candidate sample data that meets the preset condition from the second image set through the image processing model trained for the Nth round, the training unit 501 is used to: perform target detection on the second image set through the image processing model trained for the Nth round to obtain sample data in the second image set that does not contain the target to be detected; and determine the candidate sample data as sample data that does not contain the target to be detected.

[0118] According to one or more embodiments of this disclosure, in the process of updating the first image set using candidate sample data, the training unit 501 is used to: add a target to be detected to the candidate sample data to obtain updated candidate sample data; and update the first image set according to the updated candidate sample data.

[0119] According to one or more embodiments of this disclosure, in the process of adding a target to be detected to candidate sample data to obtain updated candidate sample data, the training unit 501 is used to: use the candidate sample data as a background image, synthesize the sample data in the third image set with the candidate sample data to obtain updated candidate sample data, wherein the sample data in the third image set is a mask image of the target to be detected.

[0120] According to one or more embodiments of this disclosure, during the process of updating the first image set based on updated candidate sample data, the training unit 501 is configured to: determine the updated candidate sample data as newly added sample data in the first image set; or, perform data augmentation processing on the updated candidate sample data, and determine the data-augmented candidate sample data as newly added sample data in the first image set.

[0121] The image processing model training device provided in this embodiment can be used to execute the technical solution of the above-described image processing model training method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0122] Corresponding to the image processing method in the above embodiments, Figure 6 This is a structural block diagram of an image processing apparatus provided according to embodiments of the present disclosure. For ease of explanation, only the parts relevant to embodiments of the present disclosure are shown. (Refer to...) Figure 6 The image processing device 600 includes an acquisition unit 601 and a prediction unit 603.

[0123] Acquisition unit 601 is used to acquire the target image;

[0124] The prediction unit 602 is used to predict whether a target to be detected exists in the target image using an image processing model, and obtain a prediction result. The image processing model is trained using the image processing model training method provided in any of the foregoing embodiments.

[0125] The image processing device provided in this embodiment can be used to execute the technical solutions of the above-described image processing method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0126] refer to Figure 7 The diagram illustrates a structural schematic of an electronic device 700 suitable for implementing embodiments of the present disclosure. The electronic device 700 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0127] like Figure 7As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0128] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0129] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0130] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0131] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0132] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0133] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0135] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0136] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0137] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0138] In a first aspect, according to one or more embodiments of this disclosure, an image processing model training method is provided, comprising: training an image processing model using a first image set to obtain an image processing model after Nth round of training, where N is greater than or equal to 1, and the sample data in the first image set is labeled data; selecting candidate sample data that meet preset conditions from a second image set using the image processing model after Nth round of training, where the sample data in the second image set is unlabeled data; updating the first image set using the candidate sample data; and training the image processing model after Nth round of training based on the updated first image set to obtain an image processing model after N+1th round of training.

[0139] According to one or more embodiments of this disclosure, after training the image processing model after the Nth round of training based on the updated first image set to obtain the image processing model after the N+1th round of training, the method further includes: terminating the training of the image processing model in response to the number of training rounds of the image processing model being greater than a round number threshold or the screening accuracy of the image processing model being greater than an accuracy threshold.

[0140] According to one or more embodiments of this disclosure, the preset condition is that there is no target to be detected in the sample data. The candidate sample data that meets the preset condition is selected from the second image set by the image processing model after the Nth round of training. This includes: performing target detection on the second image set by the image processing model after the Nth round of training to obtain sample data in the second image set that does not contain the target to be detected; and determining the candidate sample data as sample data that does not contain the target to be detected.

[0141] According to one or more embodiments of this disclosure, updating a first image set using candidate sample data includes: adding a target to be detected to the candidate sample data to obtain updated candidate sample data; and updating the first image set based on the updated candidate sample data.

[0142] According to one or more embodiments of this disclosure, adding a target to be detected to candidate sample data to obtain updated candidate sample data includes: using the candidate sample data as a background image, performing image synthesis between the sample data in a third image set and the candidate sample data to obtain updated candidate sample data, wherein the sample data in the third image set is a mask image of the target to be detected.

[0143] According to one or more embodiments of this disclosure, updating a first image set based on updated candidate sample data includes: determining the updated candidate sample data as new sample data in the first image set; or, performing data augmentation processing on the updated candidate sample data, and determining the data-augmented candidate sample data as new sample data in the first image set.

[0144] In a second aspect, according to one or more embodiments of the present disclosure, an image processing method is provided, comprising: acquiring a target image; predicting whether a target to be detected exists in the target image using an image processing model, and obtaining a prediction result, wherein the image processing model is trained according to the image processing model training method provided in the first aspect and / or any embodiment of the first aspect.

[0145] Thirdly, according to one or more embodiments of this disclosure, an image processing model training device is provided, comprising: a training unit, configured to train an image processing model using a first image set to obtain an image processing model after Nth round of training, where N is greater than or equal to 1, and the sample data in the first image set is labeled data; a filtering unit, configured to filter candidate sample data that meets preset conditions from a second image set using the image processing model after Nth round of training, where the sample data in the second image set is unlabeled data; an updating unit, configured to update the first image set using the candidate sample data; and a training unit, further configured to train the image processing model after Nth round of training based on the updated first image set to obtain an image processing model after N+1th round of training.

[0146] According to one or more embodiments of this disclosure, the training unit is further configured to: terminate the training of the image processing model in response to the number of training rounds of the image processing model being greater than a round number threshold or the screening accuracy of the image processing model being greater than an accuracy threshold.

[0147] According to one or more embodiments of this disclosure, the preset condition is that there is no target to be detected in the sample data. In the process of selecting candidate sample data that meet the preset condition from the second image set through the image processing model trained for the Nth round, the training unit is used to: perform target detection on the second image set through the image processing model trained for the Nth round to obtain sample data in the second image set that does not contain the target to be detected; and determine the candidate sample data as sample data that does not contain the target to be detected.

[0148] According to one or more embodiments of this disclosure, in the process of updating the first image set using candidate sample data, the training unit is configured to: add a target to be detected to the candidate sample data to obtain updated candidate sample data; and update the first image set based on the updated candidate sample data.

[0149] According to one or more embodiments of this disclosure, in the process of adding a target to be detected to candidate sample data to obtain updated candidate sample data, the training unit is used to: use the candidate sample data as a background image, synthesize the sample data in the third image set with the candidate sample data to obtain updated candidate sample data, wherein the sample data in the third image set is a mask image of the target to be detected.

[0150] According to one or more embodiments of this disclosure, during the process of updating the first image set based on updated candidate sample data, the training unit is configured to: determine the updated candidate sample data as newly added sample data in the first image set; or, perform data augmentation processing on the updated candidate sample data, and determine the data-augmented candidate sample data as newly added sample data in the first image set.

[0151] Fourthly, according to one or more embodiments of the present disclosure, an image processing apparatus is provided, comprising: an acquisition unit for acquiring a target image; and a prediction unit for predicting whether a target to be detected exists in the target image using an image processing model, and obtaining a prediction result, wherein the image processing model is trained according to the image processing model training method provided in any embodiment of the first aspect.

[0152] Fifthly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory;

[0153] The memory stores the instructions that the computer executes;

[0154] At least one processor executes computer execution instructions stored in memory, causing the at least one processor to perform the image processing model training method provided in any embodiment of the first aspect above, or to perform the image processing method provided in any embodiment of the second aspect.

[0155] In a sixth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, which stores computer-executable instructions that, when executed by a processor, implement the image processing model training method provided in any embodiment of the first aspect above, or implement the image processing method provided in any embodiment of the second aspect.

[0156] In a seventh aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, the computer program product including computer execution instructions, which, when executed by a processor, implement the image processing model training method provided in any embodiment of the first aspect, or implement the image processing method provided in any embodiment of the second aspect.

[0157] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0158] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0159] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for training an image processing model, comprising: The image processing model is trained using the first image set to obtain the image processing model after the Nth round of training, where N is greater than or equal to 1, and the sample data in the first image set is labeled data. The image processing model trained in the Nth round is used to select candidate sample data that meet the preset conditions from the second image set. The sample data in the second image set is unlabeled data. The first image set is updated using the candidate sample data; Based on the updated first image set, the image processing model after the Nth round of training is trained to obtain the image processing model after the N+1th round of training. The preset condition is that there is no target to be detected in the sample data. The step of selecting candidate sample data that meets the preset condition from the second image set through the image processing model trained in the Nth round includes: Using the image processing model trained in the Nth round, target detection is performed on the second image set to obtain sample data in the second image set that do not contain the target to be detected; The candidate sample data is determined to be the sample data in which no target to be detected exists; Updating the first image set includes: The target to be detected is added to the candidate sample data to obtain updated candidate sample data; The first image set is updated based on the updated candidate sample data.

2. The method according to claim 1, further comprising, after training the image processing model after the Nth round of training based on the updated first image set to obtain the image processing model after the N+1th round of training: The training of the image processing model ends when the number of training epochs of the image processing model exceeds a threshold or the screening accuracy of the image processing model exceeds an accuracy threshold.

3. The method according to claim 1, wherein adding the target to be detected to the candidate sample data to obtain updated candidate sample data comprises: Using the candidate sample data as a background image, the sample data in the third image set is combined with the candidate sample data to obtain the updated candidate sample data. The sample data in the third image set is a mask image of the target to be detected.

4. The method according to claim 1, wherein updating the first image set based on the updated candidate sample data comprises: The updated candidate sample data is determined as the new sample data in the first image set; Alternatively, data augmentation processing can be performed on the updated candidate sample data, and the data augmented candidate sample data can be determined as new sample data in the first image set.

5. An image processing method, characterized in that, include: Acquire the target image; The image processing model is used to predict whether there is a target to be detected in the target image, and the prediction result is obtained. The image processing model is trained by the image processing model training method according to any one of claims 1 to 4.

6. An image processing model training device, comprising: The training unit is used to train the image processing model using the first image set to obtain the image processing model after the Nth round of training, where N is greater than or equal to 1, and the sample data in the first image set is labeled data. The filtering unit is used to filter candidate sample data that meet preset conditions from the second image set through the image processing model trained in the Nth round. The sample data in the second image set is unlabeled data. An update unit is used to update the first image set using the candidate sample data; The training unit is also used to train the image processing model after the Nth round of training based on the updated first image set to obtain the image processing model after the N+1th round of training. The preset condition is that there is no target to be detected in the sample data. The filtering unit is specifically used to perform target detection on the second image set through the image processing model trained in the Nth round, to obtain sample data in the second image set that does not contain the target to be detected; and to determine the candidate sample data as the sample data that does not contain the target to be detected. The update unit is specifically used to add the target to be detected to the candidate sample data to obtain updated candidate sample data; and to update the first image set according to the updated candidate sample data.

7. An image processing apparatus, comprising: The acquisition unit is used to acquire the target image; The prediction unit is used to predict whether there is a target to be detected in the target image by using an image processing model, and to obtain a prediction result, wherein the image processing model is trained by the image processing model training method according to any one of claims 1 to 4.

8. An electronic device, comprising: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1 to 5.

9. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 5.

Images