Model training, image processing method, device, equipment and storage medium

Abstract

The present disclosure relates to a model training method and device, an image processing method and device, an equipment and a storage medium. The present disclosure trains a model by using a first data set containing multi-modal information such as image information and text information, and obtains a model after a first round of iterative training. The model after the first round of iterative training is used as a benchmark for multiple rounds of iterative training, and a pre-trained model is obtained. Before each round of iterative training, the target data set used in the last round of iterative training is updated. According to the updated target data set, the model after the last round of iterative training is subjected to the current round of iterative training, so that the model after the current round of iterative training is more accurate than the model after the last round of iterative training. The pre-trained model is used as a starting model for a downstream task such as a scrap steel grade determination task, and the pre-trained model is fine-tuned according to sample data in the downstream task, so that the fine-tuned pre-trained model accurately determines the grade of scrap steel, thereby effectively saving scrap steel resources.

Description

Technical Field

[0001] This disclosure relates to the field of information technology, and in particular to a model training, image processing method, apparatus, device, and storage medium. Background Technology

[0002] Currently, image processing technology has been widely applied in various fields. For example, by acquiring images of scrap steel from a complete vehicle, and then using image processing technology, the grade of the scrap steel can be determined.

[0003] However, current image processing technology is not accurate enough in determining the grade of scrap steel from vehicles, resulting in the ineffective utilization of scrap steel resources. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a model training, image processing method, apparatus, device, and storage medium to accurately determine the grade of scrap steel and save scrap steel resources.

[0005] In a first aspect, embodiments of this disclosure provide a model training method, including:

[0006] The model after the previous iteration training and the target dataset used in the previous iteration training are obtained. The target dataset includes at least a first dataset, which includes multiple first samples. The first sample includes a first image and text information of the target object in the first image. The first dataset is used for model training to obtain the model after the first iteration training. The model after the previous iteration training is the model after the first iteration training, or it is a model trained by at least one iteration training based on the model after the first iteration training.

[0007] The target dataset is updated using the model trained in the previous iteration.

[0008] Based on the updated target dataset, the model trained in the previous iteration is iteratively trained to obtain the model trained in the current iteration. After multiple iterations of training, a pre-trained model is obtained.

[0009] Secondly, embodiments of this disclosure provide a model training method, the method comprising:

[0010] Obtain the first and second images from the sample dataset;

[0011] Based on the machine learning model to be trained, the first image and the second image are detected respectively to obtain a first detection result of the first image and a second detection result of the second image. The initial parameters of the machine learning model to be trained are the parameters of the pre-trained model, which is obtained according to the method described in the first aspect.

[0012] Multiple first candidate regions in the first image and multiple second candidate regions in the second image are fused together to obtain multiple third candidate regions;

[0013] Based on the plurality of third candidate regions, a third detection result is obtained, wherein the first detection result, the second detection result, and the third detection result each include at least one target region and the category of the target object in each target region;

[0014] Based on the first detection result, the second detection result, and the third detection result, the machine learning model to be trained is trained to obtain the trained machine learning model.

[0015] Thirdly, embodiments of this disclosure provide an image processing method, the method comprising:

[0016] Obtain the target image to be processed;

[0017] The target image is input into a trained machine learning model, such that the trained machine learning model outputs at least one target region in the target image and the confidence level of the category to which the target object in each target region belongs, wherein the trained machine learning model is obtained according to the method described in the second aspect;

[0018] The level of the target object in the target image is determined based on the confidence level of the category to which the target object belongs in each target region.

[0019] Fourthly, embodiments of this disclosure provide a method for determining the grade of scrap steel, the method comprising:

[0020] Acquire images of the entire vehicle's scrap steel during the unloading process;

[0021] The image at each level is input into the trained machine learning model, such that the trained machine learning model outputs at least one target region in the image at each level, and the category of the target object in each target region, wherein the trained machine learning model is obtained according to the method described in the second aspect;

[0022] The grade of the scrap steel from the vehicle is determined based on at least one target region in the image of each level and the category of the target object in each target region.

[0023] Fifthly, embodiments of this disclosure provide a model training apparatus, comprising:

[0024] The acquisition module is used to acquire the model after the previous round of iteration training and the target dataset used in the previous round of iteration training. The target dataset includes at least a first dataset, which includes multiple first samples. The first sample includes a first image and text information of the target object in the first image. The first dataset is used for model training to obtain the model after the first round of iteration training. The model after the previous round of iteration training is the model after the first round of iteration training, or a model after at least one round of iteration training based on the model after the first round of iteration training.

[0025] The update module is used to update the target dataset using the model trained in the previous iteration.

[0026] The iterative training module is used to iteratively train the model after the previous iteration based on the updated target dataset to obtain the model after the current iteration. After multiple iterations of training, a pre-trained model is obtained.

[0027] Sixthly, embodiments of this disclosure provide an electronic device, including:

[0028] Memory;

[0029] Processor; and

[0030] Computer programs;

[0031] The computer program is stored in the memory and configured to be executed by the processor to implement the methods described in the first, second, third, and fourth aspects.

[0032] In a seventh aspect, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the methods described in the first aspect, the second aspect, the third aspect, and the fourth aspect.

[0033] The model training, image processing method, apparatus, device, and storage medium provided in this disclosure are used to train a model using a first dataset containing multimodal information such as image information and text information, resulting in a model after the first round of iterative training. Further, multiple rounds of iterative training are performed using the model after the first round of iterative training as a benchmark to obtain a pre-trained model. Before each round of iterative training, the target dataset used in the previous round of iterative training is updated, so that the updated target dataset contains more samples. Therefore, when iteratively training the model after the previous round of iterative training based on the updated target dataset, the model after the current round of iterative training is more accurate than the model after the previous round of iterative training. In other words, during the multiple rounds of iterative training, the accuracy of the model continuously improves with the increase in the number of rounds, and the accuracy of the pre-trained model obtained after multiple rounds of iterative training reaches its highest level. Therefore, using this pre-trained model as the starting model for a downstream task, such as scrap steel grade determination, and fine-tuning the pre-trained model based on sample data from the downstream task, the fine-tuned pre-trained model can accurately determine the scrap steel grade, thereby effectively saving scrap steel resources. Attached Figure Description

[0034] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0035] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0036] Figure 1 A flowchart of the model training method provided in this embodiment of the disclosure;

[0037] Figure 2 A schematic diagram illustrating an application scenario provided by an embodiment of this disclosure;

[0038] Figure 3 A schematic diagram illustrating text information provided in embodiments of this disclosure;

[0039] Figure 4 A schematic diagram of text annotation provided for embodiments of this disclosure;

[0040] Figure 5 A flowchart of a model training method provided in another embodiment of this disclosure;

[0041] Figure 6 A flowchart of a model training method provided in another embodiment of this disclosure;

[0042] Figure 7 A flowchart of a model training method provided in another embodiment of this disclosure;

[0043] Figure 8 A self-annotated schematic diagram provided for another embodiment of this disclosure;

[0044] Figure 9 A schematic diagram of the annotation results provided in the embodiments of this disclosure;

[0045] Figure 10 A schematic diagram illustrating the annotation results provided in another embodiment of this disclosure;

[0046] Figure 11 A schematic diagram of a semi-supervised training method provided in an embodiment of this disclosure;

[0047] Figure 12 A flowchart of a model training method provided in another embodiment of this disclosure;

[0048] Figure 13 A flowchart of a model training method provided in another embodiment of this disclosure;

[0049] Figure 14 A flowchart of a model training method provided in another embodiment of this disclosure;

[0050] Figure 15 A flowchart of a model training method provided in another embodiment of this disclosure;

[0051] Figure 16 A schematic diagram of a GDL provided for another embodiment of this disclosure;

[0052] Figure 17 A schematic diagram of a PCB provided for another embodiment of this disclosure;

[0053] Figure 18 This is a flowchart of an image processing method provided in another embodiment of the present disclosure;

[0054] Figure 19 A flowchart of a scrap steel grade determination method provided in another embodiment of this disclosure;

[0055] Figure 20 Here is a flowchart of another method for determining the grade of scrap steel provided in this disclosure;

[0056] Figure 21 A schematic diagram of the user interface provided in this disclosure;

[0057] Figure 22 This is a schematic diagram of the structure of the model training device provided in the embodiments of this disclosure;

[0058] Figure 23This is a schematic diagram of the structure of the model training device provided in the embodiments of this disclosure;

[0059] Figure 24 This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this disclosure;

[0060] Figure 25 This is a schematic diagram of the scrap steel grade determination device provided in the embodiments of this disclosure;

[0061] Figure 26 A schematic diagram of the structure of an electronic device embodiment provided in this disclosure. Detailed Implementation

[0062] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0063] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0064] It should be noted that the images (including but not limited to images captured by user equipment, images obtained by user equipment from the network, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0065] In addition, the model training method, image processing method, and scrap steel grade determination method provided in this application involve the following explanations of terms, detailed below:

[0066] NLP: Natural Language Processing.

[0067] NMS: Non-maximum suppression.

[0068] Logit (also known as "rating model", "classification rating model", or "logistic regression") is one of the discrete choice models.

[0069] Transformer: A model that uses attention mechanisms to improve the training speed of a model.

[0070] Typically, images of the entire vehicle's scrap steel are acquired, and then image processing technology is used to determine the scrap steel's grade. However, current image processing technology for determining the grade of entire vehicle's scrap steel is not accurate enough, leading to ineffective utilization of scrap steel resources. To address this problem, this disclosure provides a model training method, which will be described below with reference to specific embodiments.

[0071] Figure 1 This is a flowchart illustrating a model training method provided in an embodiment of this disclosure. The method can be executed by a model training device, which can be implemented in software and / or hardware. This device can be configured in an electronic device, such as a server or terminal, where the terminal specifically includes a mobile phone, computer, or tablet computer. The server can specifically be a cloud server, and the model training method can be executed in the cloud. Several computing nodes (cloud servers) can be deployed in the cloud, each with computing, storage, and other processing resources. In the cloud, multiple computing nodes can be organized to provide a certain service; of course, a single computing node can also provide one or more services. The cloud can provide the service by providing a service interface, which users call to use the corresponding service. Service interfaces include Software Development Kits (SDKs), Application Programming Interfaces (APIs), etc. Furthermore, the model training method described in this embodiment is applicable to... Figure 2 The application scenarios shown are as follows. Figure 2 As shown, this application scenario includes terminal 21 and server 22, where server 22 can perform multiple rounds of iterative training to obtain a pre-trained model. The following section combines... Figure 2 This method will be described in detail, such as Figure 1 As shown, the specific steps of this method are as follows:

[0072] S101. Obtain the model after the previous iteration training and the target dataset used in the previous iteration training. The target dataset includes at least a first dataset, which includes multiple first samples. The first sample includes a first image and text information of the target object in the first image. The first dataset is used for model training to obtain the model after the first iteration training. The model after the previous iteration training is the model after the first iteration training, or a model trained by at least one iteration based on the model after the first iteration training.

[0073] For example, server 22 can obtain a first dataset, denoted as dataset A. Dataset A may include multiple first samples, each first sample including a first image and text information about a target object in that first image. The text information about the target object in the first image may be a textual description of the target object in the first image. For example... Figure 3 As shown, the first image 31 is an image of the scrap steel after image acquisition, that is, the target object 32 in the first image 31 is the scrap steel. The text information corresponding to the first image 31 can be a text description of the target object 32, such as the color, surface texture, size, shape, thickness, etc. of the target object 32.

[0074] It is understood that the target object included in the first image 31 is not limited to one. For example, the first image 31 may include multiple target objects, which are not limited to scrap steel, but may also include other cement blocks, soil blocks, etc. Each target object may correspond to a text description. When the first image 31 includes multiple target objects, the first image 31 and the text descriptions corresponding to the multiple target objects constitute a first sample. Furthermore, as... Figure 3 The first image 31 and text information shown are a schematic diagram of a first sample. The construction process of other first samples is similar to that of this first sample, and will not be described in detail here. Furthermore, this embodiment does not limit the method by which the server 22 acquires the first image. For example, the terminal 21 can capture an image of scrap steel and then send the captured image, such as the first image, to the server 22. Alternatively, the server 22 can acquire the first image containing scrap steel from other terminals or other servers. In addition, the text information corresponding to the first image can be obtained through analysis by the server 22, or it can be manually pre-annotated.

[0075] Furthermore, server 22 can perform one round of iterative training on a pre-defined machine learning model using multiple first samples from dataset A. For example, this pre-defined machine learning model could be a model with initial parameters of 0, or a model that has not been pre-trained or fine-tuned. In this round of iterative training, the process of training the pre-defined machine learning model once using a single first sample is denoted as one iteration; that is, one round of iterative training includes multiple iterations. The following uses… Figure 3 Taking an example, we will introduce the process of one iteration of training. For example, server 22 can... Figure 3The first image 31 and text information shown are input into the preset machine learning model, enabling the preset machine learning model to predict the category to which the target object 32 belongs. For example, the preset machine learning model can output four confidence scores, which represent the probabilities of the target object 32 being steel, non-steel, scrap steel, and hazardous waste, respectively. Further, the maximum value is determined from these four confidence scores, and the category corresponding to this maximum value can be used as the category to which the target object 32 belongs, as predicted by the preset machine learning model. For example, assuming the probability that the target object 32 is scrap steel is the highest, then the category predicted by the preset machine learning model is scrap steel. Additionally, the category to which the target object 32 belongs can be pre-labeled for the first image 31, for example, manually. Based on the category to which the target object 32 belongs predicted by the preset machine learning model and the pre-labeled category to which the target object 32 belongs, a loss function is constructed, and the parameters of the preset machine learning model are updated based on this loss function, i.e., an iterative training of the preset machine learning model is performed. After one iteration of training, the parameters of the preset machine learning model are updated. Based on this, the next first sample is used to train the preset machine learning model with updated parameters for the next iteration. This process is repeated until every first sample in dataset A has participated in the iterative training, or until the number of iterations is greater than or equal to the preset number, or until the parameters of the preset machine learning model tend to converge. This results in the model after the first round of iterative training, which is denoted as the first machine learning model.

[0076] For example, based on the similarity or difference between the category of the target object predicted by the preset machine learning model and the category of the pre-labeled target object, the parameters in the preset machine learning model can be adjusted so that the categories of the target object predicted by the preset machine learning model and the categories of the pre-labeled target object gradually become similar during subsequent iterative training. For instance, the parameter adjustment can be guided by the gradient data between the categories of the target object predicted by the preset machine learning model and the categories of the pre-labeled target object. Here, the gradient is essentially a vector representing the directional derivative of a function at a given point, where the directional derivative reaches its maximum value along that direction; that is, the function changes most rapidly and has the largest rate of change along that direction (the direction of the gradient) at that point. It is based on this principle that the direction of parameter adjustment can be guided so that the category of the target object predicted by the preset machine learning model moves closer to the category of the pre-labeled target object.

[0077] Furthermore, the method of training a pre-defined machine learning model using the first sample is not limited to the method described above. In other embodiments, a contrastive learning method can also be used. For example, the first sample includes a first image and text information. The first image is a scrap steel image, and the text information is descriptive text. It is understood that if the scrap steel image and the descriptive text come from the same first sample, then the scrap steel image and the descriptive text are related. If the scrap steel image and the descriptive text come from different first samples, then the scrap steel image and the descriptive text are not related. Before training, a pair of related scrap steel images A and descriptive text B can be selected, as well as a pair of unrelated scrap steel images C and descriptive text D. That is, scrap steel images A and descriptive text B are the same pair, and scrap steel images C and descriptive text D are different pairs.

[0078] Then, feature extraction is performed on scrap steel image A, descriptive text B, scrap steel image C, and descriptive text D respectively, obtaining the feature vectors corresponding to each image. For example, the backbone network for feature extraction of scrap steel images A and C can be a ResNet series model or a Vision Transformer (VIT) series model. The model for feature extraction of descriptive text B and D can be a Bidirectional Encoder Representation from Transformer (BERT) model based on a transformer. Since images and text are information of different modalities, the dimension of the feature vector of the scrap steel image may be different from that of the feature vector of the descriptive text. Therefore, in this embodiment, the feature vectors corresponding to scrap steel images A, B, C, and D can be normalized. Furthermore, the normalized feature vectors of scrap steel image A and descriptive text B are input into a pre-defined machine learning model, enabling the model to predict the category of the target object in scrap steel image A. A first loss function is constructed based on the predicted category and the pre-labeled category of the target object in scrap steel image A. Similarly, the normalized feature vectors of scrap steel image C and descriptive text D are input into the same model, again predicting the category of the target object in scrap steel image C. A second loss function is constructed based on this prediction and the pre-labeled category. Finally, a contrastive loss function (info-nce-loss) is constructed based on the first and second loss functions. The parameters of the pre-defined machine learning model are updated by controlling the value of the first loss function to approach 1 and the value of the second loss function to approach 0. In other words, the predicted result obtained by the pre-defined machine learning model based on the same pair of scrap steel images and descriptive text is very close to the actual labeled result, for example, close to 1. The predicted results obtained by the preset machine learning model based on different scrap steel images and descriptive texts are far from the actual labeled results, for example, approaching 0.

[0079] Furthermore, server 22 can obtain an unlabeled second dataset, denoted as dataset B, select a useful subset B' from dataset B, and perform the second round of iterative training on the first machine learning model based on dataset A and subset B', thus obtaining the second machine learning model.

[0080] Furthermore, server 22 can obtain an unlabeled third dataset, denoted as dataset C. It selects a useful subset C' from dataset C and uses dataset A, subset B', and subset C' to perform the next round of iterative training on the second machine learning model, i.e., the third round of iterative training, to obtain the third machine learning model. This process is repeated for multiple rounds of iterative training.

[0081] Understandably, besides the first machine learning model, before each iteration of training, the model trained in the previous iteration and the target dataset used in the previous iteration can be obtained. For example, before the second iteration, the model trained in the first iteration (i.e., the first machine learning model) and the target dataset used in the first iteration (e.g., dataset A) can be obtained. Similarly, before the third iteration, the model trained in the second iteration (i.e., the second machine learning model) and the target dataset used in the second iteration (e.g., dataset A and subset B') can be obtained. And so on. Therefore, regardless of which iteration the model trained in the previous iteration is, the target dataset used in the previous iteration at least includes the first dataset (i.e., dataset A). Furthermore, the model trained in the previous iteration may be the first machine learning model, or it may be a model trained on the first machine learning model as a baseline for at least one iteration.

[0082] S102. Update the target dataset using the model trained in the previous iteration.

[0083] S103. Based on the updated target dataset, iteratively train the model trained in the previous iteration to obtain the model trained in the current iteration. After multiple iterations, a pre-trained model is obtained.

[0084] In this embodiment, regardless of which iteration the model was trained in the previous iteration, it can be used to update the target dataset used in the previous iteration. Then, using the updated target dataset, the model trained in the previous iteration is iteratively trained to obtain the model trained in the current iteration.

[0085] For example, when the model trained in the previous iteration is the model trained in the first iteration (i.e., the first machine learning model), the target dataset used in the first iteration is dataset A. Further, the first machine learning model can select a useful subset B' from dataset B, and update dataset A based on subset B'. The updated target dataset includes dataset A and subset B'. Further, based on the updated target dataset, for example, dataset A and subset B', the first machine learning model is trained in a second iteration to obtain the model trained in this iteration (i.e., the second machine learning model).

[0086] For example, when the model trained in the previous iteration is the model trained in the second iteration (i.e., the second machine learning model), the target dataset used in the second iteration is dataset A and subset B'. Further, the second machine learning model can select a useful subset C' from dataset C, and update the target dataset used in the second iteration based on subset C'. The updated target dataset includes dataset A, subset B', and subset C'. Further, based on the updated target dataset, for example, dataset A, subset B', and subset C', the second machine learning model undergoes a third iteration of training to obtain the model trained in this iteration (i.e., the third machine learning model). And so on. In this embodiment, the model obtained after multiple iterations of training is referred to as the pre-trained model. In some embodiments, this pre-trained model can also be referred to as a knowledge-based model, a large knowledge-based model, or a pre-trained multimodal large model, etc.

[0087] Specifically, the pre-trained model can serve as the starting model for a downstream task, and can be fine-tuned based on sample data from the downstream task, so that the fine-tuned pre-trained model can be used as the model to perform the downstream task. This downstream task can be an image processing-related task, for example, a scrap steel grading task.

[0088] This embodiment of the disclosure trains a model using a first dataset containing multimodal information such as image and text information to obtain a model after the first round of iterative training. Further, multiple rounds of iterative training are performed using the model after the first round of iterative training as a benchmark to obtain a pre-trained model. Before each round of iterative training, the target dataset used in the previous round of iterative training is updated to include more samples. Therefore, when the model after the previous round of iterative training is iteratively trained based on the updated target dataset to obtain the model after the current round of iterative training, the model after the current round of iterative training is more accurate than the model after the previous round of iterative training. In other words, during the multiple rounds of iterative training, the accuracy of the model continuously improves with the increase in the number of rounds, and the accuracy of the pre-trained model obtained after multiple rounds of iterative training reaches its highest level. Therefore, using this pre-trained model as the starting model for downstream tasks, such as scrap steel grade determination, and fine-tuning the pre-trained model based on sample data from the downstream task, the fine-tuned pre-trained model can accurately determine the scrap steel grade, thereby effectively saving scrap steel resources.

[0089] In the above embodiments, since the text information corresponding to the first image can be obtained through analysis by the server 22 or can be manually pre-annotated, this embodiment provides a detailed description of the process by which the server 22 analyzes the text information. For example, the server 22 may be equipped with... Figure 4 The text description generator (image-captioning) is shown. Images 41, 42, 43, and 44 represent the first image, and 4mm, 6mm, 8mm, ..., 20mm are the image labels for images 41, 42, 43, and 44, respectively. For example, images 41, 42, 43, and 44 each include scrap steel, and 4mm could represent the thickness of most of the scrap steel in image 41, or the average thickness of the scrap steel included in image 41. The meanings of the other image labels are similar and will not be elaborated here. When image 41 is input to the text description generator, the generator can generate a text description corresponding to image 41. For example, this text description could be a text description of each piece of scrap steel included in image 41, and the text description of each piece of scrap steel would be similar to... Figure 3 The text information shown, i.e., the text description of each piece of scrap steel, can be a set of attributes of that scrap steel. Similarly, this text description generator can generate text descriptions for other images. Additionally, as... Figure 4As shown, the server 22 can also be equipped with a text encoder, which can be a Bidirectional Encoder Representation from Transformer (BERT) model. For example, the text encoder can sequentially output the feature information of the text descriptions corresponding to images 41, 42, 43, and 44. For example, T1 represents the feature information of the text description corresponding to image 41, T2 represents the feature information of the text description corresponding to image 42, and so on, with Tn representing the feature information of the text description corresponding to image 44.

[0090] Understandably, in Figure 4 In this process, the image label for each image is not limited to the thickness of the scrap steel; for example, it can also include the color and texture of the scrap steel. Furthermore, the image label can be manually annotated.

[0091] Assume the model trained in the previous iteration is the model trained in the first iteration, i.e., the first machine learning model, and the target dataset used in the first iteration is dataset A. This example describes the process of updating dataset A using the first machine learning model.

[0092] Optionally, updating the target dataset using the model trained in the previous iteration includes, for example: Figure 5 The steps shown are as follows:

[0093] S501. Using the model trained in the previous iteration, a target subset is selected from the unlabeled second dataset, and the evaluation index of each second image in the target subset meets the preset conditions.

[0094] For example, using a first machine learning model, a target subset B' can be selected from an unlabeled second dataset, such as dataset B, where the evaluation metrics of each second image in the target subset B' meet preset conditions.

[0095] S502. Using the model trained in the previous iteration, each second image in the target subset is labeled to obtain the labeled target subset.

[0096] For example, the first machine learning model can be used to label each second image in the target subset B' to obtain the labeled target subset B'.

[0097] S503. Update the target dataset according to the labeled target subset, wherein the target dataset and the labeled target subset constitute the updated target dataset.

[0098] For example, dataset A is updated based on the labeled target subset B', so that the updated target dataset includes dataset A and the labeled target subset B'.

[0099] It is understood that this embodiment uses the example of updating the target dataset used in the first round of training by the first machine learning model. However, the process of updating the target dataset used in the previous round of training by other models trained in the previous round is similar, and will not be described in detail here.

[0100] Based on the above embodiments, the target subset is selected from the unlabeled second dataset using the model trained in the previous iteration, including, for example... Figure 6 The steps shown are as follows:

[0101] S601. For each second image in the unlabeled second dataset, the second image is scored at a first granularity and a second granularity using the model trained in the previous iteration, to obtain the first granularity score and the second granularity score of the second image, wherein the first granularity is greater than the second granularity.

[0102] For example, suppose dataset B contains 100 unlabeled second images. These 100 unlabeled second images can be sequentially input into a first machine learning model, allowing the model to score each unlabeled second image from two dimensions. These two dimensions are denoted as the first dimension and the second dimension, respectively. The first dimension can be coarse-grained, and the second dimension can be fine-grained. When the first machine learning model scores the unlabeled second images using coarse-grained methods, it scores the second image based on its overall structure, obtaining a first-grained score. When the first machine learning model scores the unlabeled second images using fine-grained methods, it scores the second image based on multiple regions within it, obtaining a second-grained score. In other words, each unlabeled second image can correspond to two scores: a first-grained score and a second-grained score.

[0103] S602. Based on the first granularity score and the second granularity score corresponding to each second image, determine the evaluation index corresponding to each second image.

[0104] For example, for each unlabeled second image, the first granularity score and the second granularity score corresponding to the second image can be fused to obtain the evaluation index of the second image, i.e., the final score.

[0105] S603. The second images in the second dataset that meet the preset conditions for the evaluation index are used to form the target subset.

[0106] For example, after the above steps, each of the 100 unlabeled second images in dataset B can correspond to an evaluation metric. Further, second images with evaluation metrics greater than or equal to a preset value can be selected from these 100 unlabeled second images to form the target subset B'. Alternatively, based on the evaluation metrics corresponding to the 100 unlabeled second images, the 100 unlabeled second images can be sorted in descending order of evaluation metrics, and then the top preset number of second images in this sorted sequence can be selected to form the target subset B'.

[0107] The scoring processes for coarse-grained and fine-grained scores are described below.

[0108] Specifically, in the coarse-grained scoring process, the area under the margin (AUM) algorithm can be used to calculate the first granular score for each second image. For example, 100 unlabeled second images in dataset B can be divided into multiple min-batches, each containing multiple second images. For instance, the 100 unlabeled second images can be divided into 20 min-batches, each containing 5 second images. The first machine learning model can be fed into a single min-batch, meaning 5 second images are input together. The first machine learning model can output a predicted value (logits) for each of the 5 second images. When these 20 min-batches are sequentially input into the first machine learning model, 100 predicted values ​​are obtained. Further, the maximum value can be selected from these 100 predicted values. Then, the difference (area) between each of the 100 predicted values ​​and the maximum value is calculated, thus obtaining the difference for each of the 100 unlabeled second images. Furthermore, each of the 20 min-batches is sequentially input into the first machine learning model, and this is considered one round. In other embodiments, the 20 min-batches can be input into the first machine learning model multiple times. In this case, for each round, the difference value corresponding to each of the 100 unlabeled second images can be obtained. That is, the same second image can be input into the first machine learning model multiple times, and the difference value corresponding to the second image can be obtained after each input. Further, for each second image, the average of the differences obtained after each input is calculated to obtain the first granularity score of the second image. For example, if the 20 min-batches are input into the first machine learning model for 5 rounds, that is, the same second image will be input into the first machine learning model 5 times, and the difference value corresponding to the second image can be obtained after each input. That is, the same second image has 5 difference values. The average of these 5 differences can be used as the first granularity score of the second image. In addition, this average value can be recorded as the AUM value. Experiments showed that when the second image is relatively clean, for example, an image with less noise, the corresponding AUM value of the second image will be relatively large. When the second image is noisy, the corresponding AUM value of the second image will be relatively small, or even negative. This embodiment can separate clean images from noisy images using the AUM value. Furthermore, in dataset B, the proportion of clean images can be 99%.

[0109] In the fine-grained scoring process, 100 unlabeled second images from dataset B can be sequentially input into the first machine learning model. The first machine learning model can predict the regions included in each second image. Taking a single second image as an example, its second-granularity score can be expressed as follows:

[0110]

[0111] Where N represents the total number of regions actually included in the second image. m1 represents the number of regions missed by the first machine learning model. For example, if the second image actually includes 5 regions, and the first machine learning model detects 4 regions, then the number of missed regions is 1. m2 represents the number of regions falsely detected by the first machine learning model. For example, if the second image actually includes 5 regions, and each of these 5 regions contains 2mm of scrap steel, and the first machine learning model detects 5 regions, where 4 regions contain 2mm of scrap steel and 1 region contains 3mm of scrap steel, then the number of falsely detected regions is 1. m3 represents the number of regions misidentified by the first machine learning model. For example, if the second image actually includes 5 regions, but the first machine learning model misidentifies some background elements as regions, resulting in the first machine learning model detecting 6 regions, then the number of misidentified regions is 1. len represents a function with m1, m2, or m3 as parameters.

[0112] Through the coarse-grained and fine-grained scoring processes described above, each second image in dataset B can be scored using both coarse and fine-grained methods, resulting in two scores for each second image: a first-grained score and a second-grained score. Further, for each unlabeled second image, the first and second-grained scores can be fused to obtain the evaluation index, i.e., the final score. Then, from these 100 unlabeled second images, images with evaluation indices greater than or equal to a preset value are selected to form the target subset B'. Furthermore, each second image in the target subset B' can be labeled using a first machine learning model; that is, each second image in the target subset B' is self-labeled. This labeling process is described in detail below.

[0113] Optionally, each second image in the target subset is labeled using the model trained in the previous iteration, including, for example... Figure 7 The steps shown are as follows:

[0114] S701. For each second image in the target subset, detect at least one target region in the second image and the category of the target object in each target region using the model trained in the previous iteration.

[0115] For example, in this embodiment, multiple detection heads can be added to the first machine learning model, such as... Figure 8 As shown, detection head 1 and detection head 2 are added to the first machine learning model. For each unlabeled second image in the target subset B', the second image is input into the first machine learning model. Furthermore, each detection head on the first machine learning model can perform detection on the second image. For example, detection head 1 and detection head 2 can respectively detect at least one target region in the second image and the category of the target object in each target region. Additionally, each detection head can output a detection result; for example, detection head 1 outputs detection result 1, and detection head 2 outputs detection result 2. For example, detection result 1 includes at least one target region in the second image and the category of the target object in each target region, and detection result 2 includes at least one target region in the second image and the category of the target object in each target region. Since detection head 1 and detection head 2 are two different detection heads, detection result 1 and detection result 2 may not be exactly the same.

[0116] For example, Figure 8 The second image shown is Figure 9 As shown in image 90, detection head 1 can detect target object 92 in image 90. Specifically, the detection result 1 includes the coordinate position of target region 91 in image 90 (e.g., the coordinates of the four vertices in target region 91) and the category of target object 92 in target region 91. This category can be steel, non-steel, scrap steel, or hazardous waste. Alternatively, when target object 92 is scrap steel, the category can be the specific thickness of target object 92, such as 2mm, 4mm, or 6mm. Assuming that detection head 2 can detect not only target object 92 in image 90, but also target object 93 in image 90, such as... Figure 10 As shown, specifically, the detection result 2 includes not only the coordinate position of the target region 91 in image 90 and the category of the target object 92 in the target region 91, but also the coordinate position of the target region 94 in image 90 and the category of the target object 93 in the target region 94. For example, the target object 92 is 2mm scrap steel, and the target object 93 is a non-steel material, such as a clod of earth.

[0117] S702. Label the second image according to the category of the target object included in each target region in the second image.

[0118] For example, in this embodiment, detection result 1 and detection result 2 can be fused to obtain the annotation information of the second image, which can be recorded as a pseudo-label. Similarly, in this embodiment, a first machine learning model can be used to annotate each second image in the target subset B'.

[0119] Furthermore, based on dataset A and the labeled target subset B', the first machine learning model is trained in a second round of iterations to obtain the second machine learning model. This training process can be a semi-supervised training process. That is, the process of training the model to obtain the first machine learning model based on dataset A can be a supervised training process, and each subsequent round of iteration can be a semi-supervised training process. The following section combines... Figure 11 This section introduces the semi-supervised training process.

[0120] For example, Figure 11 The image shown is either the first labeled image in dataset A or the second labeled image in target subset B'. The teacher network can be the first machine learning model to be trained, and the student network can be a pre-trained model. The teacher and student networks respectively include a backbone network, a Feature Pyramid Network (FPN), and a Region Proposal Network (RPN). The RPN, also known as a region generation network or region candidate network, is used to extract candidate bounding boxes.

[0121] Multiple headers can be added to the teacher network, such as Teacher Head 1 and Teacher Head 2. Similarly, multiple headers can be added to the student network, such as Student Head 1 and Student Head 2. These headers can be Region of Interest (ROI) headers. For example, Teacher Head 1 is denoted as Teacher ROI Head_1, Teacher Head 2 as Teacher ROI Head_2, Student Head 1 as StudentROI Head_1, and Student Head 2 as Student ROI Head_2.

[0122] Furthermore, the image is input into both the teacher network and the student network. Teacher detection head 1 outputs current detection result 1 (Up-to-date Detection Results_1), and teacher detection head 2 outputs current detection result 2 (Up-to-date Detection Results_2). Additionally, this embodiment can also store historical detection results based on a memory mechanism. For example, the same image can be input into the teacher network multiple times; after each input, teacher detection head 1 can output a detection result, and teacher detection head 2 can output a detection result. Assume... Figure 11 The image in the image is being input into the teacher network for the second time. Current detection result 1 is the detection result output by teacher detection head 1 after this input. Current detection result 2 is the detection result output by teacher detection head 2 after this input. Historical detection result 1 is the detection result output by teacher detection head 1 when the image was first input into the teacher network, and historical detection result 2 is the detection result output by teacher detection head 2 when the image was first input into the teacher network. Historical detection result 1 can be stored in pseudo-label memory 1 (Pseudo Labels Memory_1), and historical detection result 2 can be stored in pseudo-label memory 2 (Pseudo Labels Memory_2).

[0123] Furthermore, the current detection result 1 and the historical detection result 1 are fused to obtain a first fusion result. This first fusion result can be used to update the historical detection result 1, for example, replacing the historical detection result 1 with the first fusion result. Similarly, the current detection result 2 and the historical detection result 2 are fused to obtain a second fusion result. This second fusion result can be used to update the historical detection result 2, for example, replacing the historical detection result 2 with the second fusion result. And so on, as the number of image inputs increases, the accuracy of the current detection result 1 and the current detection result 2 continuously improves. Additionally, as... Figure 11 As shown, when the image is input into the student network, student detection head 1 and student detection head 2 can output detection results respectively. Further, a first loss function is constructed based on the detection result of student detection head 1 and the first fusion result described above. A second loss function is constructed based on the detection result of student detection head 2 and the second fusion result described above. Then, the teacher network is trained based on the first and second loss functions, i.e., the parameters of the teacher network are updated. In other embodiments, the second loss function can also be constructed based on the detection result of student detection head 1 and the second fusion result described above, and the first loss function can be constructed based on the detection result of student detection head 2 and the first fusion result described above.

[0124] Understandably, during iterative training of the first machine learning model, Figure 11 The teacher network shown is the first machine learning model. During iterative training of the second machine learning model, Figure 11 The teacher network shown is a second machine learning model, meaning that the teacher network can be a model to be trained.

[0125] This embodiment employs multiple detection heads to detect the same image, allowing the detection results from each head to complement each other. The detection result of the image is obtained based on the individual results from each head, effectively improving the accuracy of image detection. Furthermore, by storing historical detection results and merging the current and historical results to update the historical results, the instability of detection results for the same image across different iterations can be mitigated.

[0126] It is understood that a pre-trained model can be obtained after multiple rounds of iterative training as described in the above embodiments. This pre-trained model can be used as the starting model for downstream tasks, and fine-tuned based on sample data from the downstream tasks, so that the fine-tuned pre-trained model can be used as the model to execute the downstream task. The fine-tuning process is described below.

[0127] Figure 12 This is a flowchart illustrating a model training method according to another embodiment of the present disclosure. The model training method is a process of fine-tuning a pre-trained model. For example, after a cloud server, such as server 22, performs multiple rounds of iterative training to obtain a pre-trained model, server 22 can either keep the pre-trained model locally or deploy it to other terminals or other servers. If server 22 keeps the pre-trained model locally, it can perform the fine-tuning process. If server 22 deploys the pre-trained model to other terminals or other servers, the fine-tuning process can be performed by those other terminals or servers. Specifically, in this embodiment, the specific steps of the method are as follows:

[0128] S1201. Obtain the first and second images from the sample dataset.

[0129] Understandably, in some cases, the sample data for downstream tasks is limited; for example, the sample data may be a small amount of data provided by a downstream steel mill (Few-shot). To make full use of the limited sample data provided by the downstream steel mill, this embodiment can use a method based on metric learning and contrastive learning to fine-tune the pre-trained model.

[0130] This embodiment uses server 22 performing a fine-tuning process as an example. Specifically, server 22 can obtain a first image and a second image from a sample dataset containing a small amount of sample data. The first image is denoted as the main image (support image), and the second image is denoted as the query image (auxiliary image). Further, the main image and auxiliary image are input into a database such as... Figure 13 In the detection network shown, specifically, in this embodiment, the parameters of the pre-trained model can be assigned to the detection network. That is, the initial parameters of the detection network are the parameters of the pre-trained model, and the detection network serves as the object of fine-tuning. For example, the detection network can be denoted as the machine learning model to be trained. This detection network can be a classic two-stage detection network. Figure 13 As shown, the detection network includes a Weight Shared Network (131), a Region Proposal Network (RPN) (132), a Region Proposal Network (133), an Attention Region Proposal Network (RPN) (134), a Region of Interest (ROI) Feature Extractor (135), a Region of Interest (ROI) Feature Extractor (136), and a Multi-Relation Head (137).

[0131] S1202. Based on the machine learning model to be trained, the first image and the second image are detected respectively to obtain a first detection result of the first image and a second detection result of the second image. The initial parameters of the machine learning model to be trained are the parameters of the pre-trained model, which is obtained according to the method described above.

[0132] like Figure 13 As shown, input the main chart and auxiliary chart into the following format: Figure 13 Following the detection network shown, the weight-sharing network 131 processes the main image and the auxiliary image respectively. Further, the processing result of the weight-sharing network 131 on the main image is input into the region candidate network 132, causing the region candidate network 132 to output multiple candidate regions in the main image; these candidate regions in the main image are designated as the first candidate regions. Simultaneously, the processing result of the weight-sharing network 131 on the auxiliary image is input into the region candidate network 133, causing the region candidate network 133 to output multiple candidate regions in the auxiliary image; these candidate regions in the auxiliary image are designated as the second candidate regions.

[0133] Furthermore, ROI pooling is performed on multiple first candidate regions, thereby processing them into multiple ROIs. Then, ROI feature extractor 135 extracts feature information for each ROI, which can be denoted as support features. Multi-relation head 137 can determine the category of the target object in each ROI based on the feature information of each ROI. Therefore, multi-relation head 137 can output multiple target regions in the main image, and the category of the target object in each target region. The target region can be an ROI, or a region determined based on an ROI. Here, the multiple target regions in the main image, and the category of the target object in each target region, can be denoted as the first detection result.

[0134] Similarly, such as Figure 13 As shown, ROI pooling is performed on multiple second candidate regions, thus processing them into multiple ROIs. Then, an ROI feature extractor 136 extracts feature information for each ROI, which can be denoted as query features. A multi-relation head 137 can determine the category of the target object in each ROI based on the feature information of each ROI. Therefore, the multi-relation head 137 can output multiple target regions in the auxiliary image, and the category of the target object in each target region. The target region can be an ROI, or a region determined based on an ROI. Here, the multiple target regions in the auxiliary image, and the category of the target object in each target region, can be denoted as the second detection result.

[0135] S1203. The multiple first candidate regions in the first image and the multiple second candidate regions in the second image are fused to obtain multiple third candidate regions.

[0136] like Figure 13 As shown, after the region candidate network 132 outputs multiple first candidate regions in the main graph and the region candidate network 133 outputs multiple second candidate regions in the auxiliary graph, the multiple first candidate regions and multiple second candidate regions can be fused in the attention region candidate network 134 to obtain multiple third candidate regions.

[0137] S1204. Based on the plurality of third candidate regions, a third detection result is obtained, wherein the first detection result, the second detection result, and the third detection result each include at least one target region and the category of the target object in each target region.

[0138] like Figure 13As shown, after obtaining multiple third candidate regions, ROI pooling can be performed on each of these regions to process them into multiple ROIs. Then, the ROI feature extractor 136 extracts the feature information of each ROI. The multi-relation head 137 can determine the category of the target object in each ROI based on the feature information of each ROI and obtain a third detection result. This third detection result includes multiple target regions and the category of the target object in each target region. The target region can be an ROI or a region determined based on the ROI.

[0139] S1205. Based on the first detection result, the second detection result, and the third detection result, train the machine learning model to be trained to obtain the trained machine learning model.

[0140] For example, the multi-relation head 137 can construct a loss function based on the first detection result, the second detection result, and the third detection result, and then train the detection network according to the loss function to obtain the trained detection network. It is understood that the training process described in this embodiment is only one iterative training of the detection network. Under normal circumstances, when the number of iterations reaches a preset number, or when the parameters of the detection network converge, the iteration stops, and the trained detection network is obtained at this time.

[0141] In one possible implementation, training the machine learning model to be trained based on the first detection result, the second detection result, and the third detection result to obtain a trained machine learning model includes: constructing a loss function based on the difference between the annotation result of the first image and the first detection result, and the difference between the second detection result and the third detection result; and training the machine learning model to be trained based on the loss function to obtain a trained machine learning model.

[0142] like Figure 13 As shown, this embodiment can also pre-annotate the main image. For example, pre-annotate the target areas in the main image and the categories of target objects in each target area, and use the pre-annotated target areas in the main image and the categories of target objects in each target area as the annotation results of the main image.

[0143] Specifically, the multi-relation head 137 can construct a loss function based on the difference between the annotation result of the main image and the first detection result, as well as the difference between the second and third detection results. Then, the detection network is trained based on this loss function. In addition, the multi-relation head 137 can also determine whether the main image and the auxiliary image match based on this loss function.

[0144] In another possible implementation, the machine learning model to be trained is trained based on the first detection result, the second detection result, and the third detection result to obtain a trained machine learning model, including: constructing a loss function based on the difference between the feature vector corresponding to the target region in the annotation result of the first image and the feature vector corresponding to the target region in the first detection result, and the difference between the feature vector corresponding to the target region in the second detection result and the feature vector corresponding to the target region in the third detection result; and training the machine learning model to be trained based on the loss function to obtain a trained machine learning model.

[0145] In this embodiment, to enable the detection network to learn more target feature representations from a small amount of sample data, a contrastive learning approach is used to better model intra-class similarity and inter-class differences. To integrate this contrastive learning approach into the two-stage detection network, this embodiment can also add a contrastive branch to the multi-relation head 137 as described above. This contrastive branch can specifically be as follows: Figure 14 The contrastive head shown. For example... Figure 14 The backbone network (Backbone ResNet-101) and FPN shown can form Figure 13 The weight-sharing network 131 shown is as follows: Figure 14 The RPN shown includes, for example, Figure 13 The region candidate networks 132 and 133 are shown. Figure 14 The ROI feature extractors shown include, for example: Figure 13 The ROI feature extractors 135 and 136 are shown. Figure 13 The multi-relationship header 137 shown includes, for example: Figure 14 The diagram shows a Box Regressor branch, a Box Classifier branch, and a comparison head. The regression branch outputs the various target regions, such as those in the main image. The classification branch outputs the categories of the target objects, similar to those in the main image. The comparison head constructs the loss function. The regression, classification, and comparison heads operate in parallel. The comparison head uses a 1-layer multi-layer perceptron (MLP) head to encode the ROI's feature information into a 128-dimensional feature vector, and then constructs the loss function based on this feature vector.

[0146] For example, the comparison head can construct a loss function based on the difference between the feature vector corresponding to the target region in the annotation results of the main image and the feature vector corresponding to the target region in the first detection result, as well as the difference between the feature vector corresponding to the target region in the second detection result and the feature vector corresponding to the target region in the third detection result. Then, the detection network is trained based on this loss function.

[0147] Optionally, the machine learning model to be trained includes a first gradient decoupling layer, a second gradient decoupling layer, a region generation network, and a feature extractor. The first gradient decoupling layer is connected to the region generation network, and the second gradient decoupling layer is connected to the feature extractor. The first gradient decoupling layer and the second gradient decoupling layer are connected in parallel. The first gradient decoupling layer and the second gradient decoupling layer are used to decouple the parameter iteration process of the region generation network and the parameter iteration process of the feature extractor.

[0148] like Figure 15 As shown, in some other embodiments, the detection network may further include a first gradient decoupled layer (GDL) 151 and a second gradient decoupled layer 152. The first gradient decoupled layer 151 is connected to a region generation network (RPN), and the second gradient decoupled layer 152 is connected to a feature extractor. Specifically, the feature extractor may be a region with CNN features (RCNN) head that functions as a convolutional neural network (CNN). Figure 15 As shown, the first gradient decoupling layer 151 and the second gradient decoupling layer 152 are connected in parallel. During the training and fine-tuning of the detection network, as... Figure 15 The solid arrows represent the process of calculating the loss function in the detection network, while the dashed arrows represent the process of updating parameters based on the loss function. The parallel connection of the first gradient decoupling layer 151 and the second gradient decoupling layer 152 allows the parameter iteration processes of the region generation network and the feature extractor to be independent of each other; that is, the parallel connection of the first gradient decoupling layer 151 and the second gradient decoupling layer 152 decouples the parameter iteration processes of the region generation network and the feature extractor. This makes the detection network architecture a simple and effective architecture, namely Decoupled Faster RCNN (DeFRCN). In other words, this architecture can extend RCNN.

[0149] Specifically, the first gradient decoupling layer 151 or the second gradient decoupling layer 152 can be as follows: Figure 16In the GDL shown, during the calculation of the loss function, it is assumed that the output of the backbone network is denoted as x, which can be used as the input of the GDL. The output of the GDL can be Ax + B, i.e., as shown in the figure. Figure 16 Circle A shown represents the calculation process of Ax + B. During the parameter iteration process, This represents the derivative of the loss function or the derivative of the backpropagation gradient. λ represents a constant or the parameters to be trained.

[0150] In addition, this embodiment introduces a Prototypical Calibration Block (PCB) into the detection network. Specifically, the PCB can calibrate the category of the target object output by the classification branch. For example, the classification branch can output the confidence level of the category to which the target object belongs, and the PCB can calibrate the confidence level of the category to which the target object belongs output by the classification branch. That is to say, as... Figure 15 The confidence levels shown for the categories are the confidence levels after calibration. The internal structure of the PCB and the calibration implementation principle are as follows: Figure 17 As shown, specifically, a main image set (Support Set) and a secondary image set (Query Set) are selected from a small amount of sample data. Further, multiple Regions of Interest (ROIs) are extracted from each main image in the main image set and multiple ROIs from each secondary image in the secondary image set. Then, the multiple ROIs extracted from each main image and each secondary image are input into a feature extractor. The feature extractor outputs prototypes and features for each ROI. The prototypes can be the mean of the features of each ROI in the main image set after processing by the feature extractor. Since each ROI corresponds to one prototype and one ROI feature, a score can be assigned based on the prototypes and features of each ROI. Specifically, the score can be a pairwise score, i.e., a score based on both the prototype and the ROI feature. Additionally, each secondary image in the secondary image set can be input into a classification branch to obtain the confidence score of the category to which each secondary image belongs. Furthermore, the highest confidence level is selected and recorded as the maximum score. Then, this maximum score is weighted and summed with paired scores to obtain a weighted score, which is the score output by the PCB. Further, the score output by the PCB is compared with... Figure 15 The confidence score of the target object's category output shown in the classification branch is calibrated. It is understood that the PCB training process can be offline.

[0151] This embodiment improves the accuracy of detection network parameter updates by adding multi-level decoupling gradient decoupling layers and a PCB for multi-task decoupling to the detection network. The multi-level decoupling gradient decoupling layers can decouple the parameter iteration processes of the region generation network and the feature extractor. Furthermore, calibrating the confidence level of the target object's category output by the PCB further improves the accuracy of the target object's category classification.

[0152] Understandably, after the detection network is fine-tuned, the fine-tuned detection network can be used during use or inference. For example, during use or inference, the fine-tuned detection network can be used to execute image processing methods. Specifically, the fine-tuned detection network can be stored in server 22 or delivered to a customer, such as a steel mill. Since this embodiment improves the accuracy of scrap steel classification based on a pre-trained multimodal large model, and performs model self-labeling on potentially useful data mined from massive scrap steel data, the delivery cycle can be greatly shortened while ensuring high algorithm accuracy. This allows steel mill technicians to use the fine-tuned detection network for image processing, thereby identifying and classifying scrap steel in images.

[0153] Figure 18 This is a flowchart illustrating an image processing method according to another embodiment of this disclosure. For example, this embodiment can be executed by an electronic device such as a server or terminal in a steel plant, for instance, by a server in a steel plant.

[0154] In this embodiment, the specific steps of the method are as follows:

[0155] S1801. Obtain the target image to be processed.

[0156] For example, a steel plant's server can receive target images captured by the steel plant's image acquisition equipment, which could be a target image of a whole truckload of scrap steel.

[0157] S1802. Input the target image into the trained machine learning model, so that the trained machine learning model outputs at least one target region in the target image and the confidence level of the category to which the target object in each target region belongs. The trained machine learning model is obtained according to the method described above.

[0158] Assuming the fine-tuned detection network is deployed on a server in a steel plant, the server can further input the target image to be processed into the fine-tuned detection network, i.e., the trained machine learning model, so that the fine-tuned detection network can output at least one target region in the target image, and the confidence score of the category to which the target object in each target region belongs.

[0159] S1803. Determine the level of the target object in the target image based on the confidence level of the category to which the target object belongs in each target region.

[0160] For example, the server can determine the level of a target object in a target image based on the confidence level of the category to which the target object belongs in each target region. For instance, a target image may contain both scrap steel and impurities. In this case, the target region output by the fine-tuned detection network includes not only the target region for scrap steel but also the target region for impurities. Assuming the target image contains 5 target regions, 4 of which contain scrap steel and 1 containing impurities, the fine-tuned detection network can output the confidence level of the scrap steel thickness (e.g., 2mm, 4mm, etc.) for each of the 4 target regions. Furthermore, by averaging the thicknesses of the scrap steel in each of the 4 target regions, the level or grade of the scrap steel in the target image is obtained. For the 1 target region, the fine-tuned detection network can output the confidence level of the impurity category (e.g., clods of soil, plastic blocks, etc.).

[0161] Optionally, the level of the target object in the target image is determined based on the confidence level of the category to which the target object belongs in each target region, including: obtaining a first image set and a second image set associated with the target image; correcting the confidence level of the category to which the target object belongs in each target region based on the first image set and the second image set to obtain a corrected confidence level; and determining the category of the target object in the target image based on the corrected confidence level.

[0162] For example, when the detection network is Figure 15 In the structure shown, during the inference phase of the detection network, assuming that the target image described above is input into the detection network, the classification branch of the detection network can output the confidence score of the category to which the target object in the target image belongs. Simultaneously, the PCB can acquire a first image set and a second image set associated with the target image; the first image set can serve as the main image set, and the second image set can serve as the auxiliary image set. Furthermore, the PCB can... Figure 15 and Figure 17 The method shown corrects the confidence level of the category to which the target object belongs in the target image output by the classification branch. Furthermore, based on the corrected confidence level, the category of the target object in the target image is determined, improving the accuracy of the target object's category.

[0163] Figure 19 This is a flowchart illustrating a scrap steel grade determination method according to another embodiment of the present disclosure. This method can be executed by a steel plant's server. In this embodiment, the specific steps of the method are as follows:

[0164] S1901. Acquire images of multiple levels of scrap steel during the unloading process.

[0165] For example, in a steel plant, a truck might carry several tons of cargo, which could include not only scrap steel but also impurities. Specifically, during the unloading process, image acquisition equipment captures multiple layers of images. For instance, the cargo in the truck is unloaded layer by layer; at least one image is captured for each layer removed, thus forming multiple layers of images—at least one image for each layer. For example, the image acquisition equipment can take pictures from multiple angles. Furthermore, the image acquisition equipment can send these multiple layers of images to the steel plant's server.

[0166] S1902. Input the image of each level into the trained machine learning model, so that the trained machine learning model outputs at least one target region in the image of each level, and the category of the target object in each target region, wherein the trained machine learning model is obtained according to the method described above.

[0167] Furthermore, the server can input images from each level, i.e., at least one image from each level, into the fine-tuned detection network, so that the fine-tuned detection network can output at least one target region in each image of each level, and the category of the target object in each target region.

[0168] S1903. Determine the grade of the whole vehicle scrap steel based on at least one target region in the image of each level and the category of the target object in each target region.

[0169] Furthermore, the server can determine the grade of the whole vehicle of scrap steel based on at least one target region in each image at each level, and the category of the target object in each target region.

[0170] Optionally, the method further includes: determining the proportion of impurities in the scrap steel of the whole vehicle based on at least one target region in the image of each level and the category of the target object in each target region.

[0171] For example, the server can determine the proportion of impurities in the scrap steel of a whole vehicle based on at least one target region in each image at each level and the category of the target object in each target region.

[0172] Figure 20This is a flowchart of another scrap steel grading method provided in this disclosure. Specifically, this embodiment can acquire multiple images using an image acquisition device, and then analyze the target objects in each image using a large knowledge-based model, i.e., a pre-trained model. This analysis includes, for example, color analysis, surface texture analysis, size analysis, morphology analysis, and thickness analysis. Then, the large knowledge-based model determines the category of each target object, for example, whether it is steel, non-steel, scrap steel, or hazardous waste. The specific determination process is as follows: Figure 20 As shown. Further, the large model based on knowledge reorganization is fine-tuned through tasks such as determining the grade of scrap steel from a whole vehicle and analyzing depreciation and impurities. The fine-tuned large model can then be used to perform tasks such as determining the grade of scrap steel from a whole vehicle and analyzing depreciation and impurities in actual unloading scenarios. After the fine-tuned large model performs these tasks in actual unloading scenarios, the steel plant's server or terminal equipment can display a user interface based on the execution results of the corresponding tasks performed by the fine-tuned large model. This user interface is shown below. Figure 21 As shown.

[0173] Figure 22 This is a schematic diagram of the structure of a model training apparatus provided in an embodiment of this disclosure. The model training apparatus provided in this embodiment of the disclosure can execute the processing flow provided in the model training method embodiment, such as... Figure 22 As shown, the model training device 220 includes:

[0174] The acquisition module 221 is used to acquire the model after the previous round of iteration training and the target dataset used in the previous round of iteration training. The target dataset includes at least a first dataset, which includes multiple first samples. The first sample includes a first image and text information of the target object in the first image. The first dataset is used for model training to obtain the model after the first round of iteration training. The model after the previous round of iteration training is the model after the first round of iteration training, or it is a model trained by at least one round of iteration training based on the model after the first round of iteration training.

[0175] The update module 222 is used to update the target dataset using the model trained in the previous iteration.

[0176] The iterative training module 223 is used to iteratively train the model after the previous round of iterative training based on the updated target dataset to obtain the model after the current round of iterative training, and obtain a pre-trained model after multiple rounds of iterative training.

[0177] Optionally, when the update module 222 updates the target dataset using the model trained in the previous iteration, it is specifically used for:

[0178] Using the model trained in the previous iteration, a target subset is selected from the unlabeled second dataset, and the evaluation metric of each second image in the target subset meets the preset conditions.

[0179] Using the model trained in the previous iteration, each second image in the target subset is labeled to obtain the labeled target subset.

[0180] The target dataset is updated based on the labeled target subset, and the target dataset and the labeled target subset constitute the updated target dataset.

[0181] Optionally, when the update module 222 selects a target subset from the unlabeled second dataset using the model trained in the previous iteration, it is specifically used for:

[0182] For each second image in the unlabeled second dataset, the second image is scored at a first granularity and a second granularity using the model trained in the previous iteration, to obtain the first granularity score and the second granularity score of the second image, where the first granularity is greater than the second granularity.

[0183] Based on the first granularity score and the second granularity score corresponding to each second image, the evaluation index corresponding to each second image is determined.

[0184] The second images in the second dataset that meet the preset conditions for evaluation metrics constitute the target subset.

[0185] Optionally, when the update module 222 annotates each second image in the target subset using the model trained in the previous iteration, it is specifically used for:

[0186] For each second image in the target subset, at least one target region in the second image and the category of the target object in each target region are detected using the model trained in the previous iteration.

[0187] The second image is labeled according to the category of the target object included in each target region in the second image.

[0188] Figure 22 The model training apparatus of the illustrated embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0189] Figure 23 This is a schematic diagram of the structure of a model training apparatus provided in an embodiment of this disclosure. The model training apparatus provided in this embodiment of the disclosure can execute the processing flow provided in the model training method embodiment, such as... Figure 23As shown, the model training device 230 includes:

[0190] The acquisition module 231 is used to acquire the first image and the second image in the sample dataset;

[0191] The detection module 232 is used to detect the first image and the second image respectively according to the machine learning model to be trained, and obtain a first detection result of the first image and a second detection result of the second image. The initial parameters of the machine learning model to be trained are the parameters of the pre-trained model, which is obtained according to the method described above.

[0192] The fusion processing module 233 is used to perform fusion processing on multiple first candidate regions in the first image and multiple second candidate regions in the second image to obtain multiple third candidate regions;

[0193] The detection module 232 is further configured to: obtain a third detection result based on the plurality of third candidate regions, wherein the first detection result, the second detection result, and the third detection result respectively include at least one target region and the category of the target object in each target region;

[0194] The training module 234 is used to train the machine learning model to be trained based on the first detection result, the second detection result and the third detection result, so as to obtain the trained machine learning model.

[0195] Optionally, when the training module 234 trains the machine learning model to be trained based on the first detection result, the second detection result, and the third detection result to obtain the trained machine learning model, it is specifically used for:

[0196] A loss function is constructed based on the difference between the annotation result of the first image and the first detection result, and the difference between the second detection result and the third detection result;

[0197] The machine learning model to be trained is trained according to the loss function to obtain the trained machine learning model.

[0198] Optionally, when the training module 234 trains the machine learning model to be trained based on the first detection result, the second detection result, and the third detection result to obtain the trained machine learning model, it is specifically used for:

[0199] A loss function is constructed based on the difference between the feature vector corresponding to the target region in the annotation result of the first image and the feature vector corresponding to the target region in the first detection result, as well as the difference between the feature vector corresponding to the target region in the second detection result and the feature vector corresponding to the target region in the third detection result.

[0200] The machine learning model to be trained is trained according to the loss function to obtain the trained machine learning model.

[0201] Optionally, the machine learning model to be trained includes a first gradient decoupling layer, a second gradient decoupling layer, a region generation network, and a feature extractor. The first gradient decoupling layer is connected to the region generation network, and the second gradient decoupling layer is connected to the feature extractor. The first gradient decoupling layer and the second gradient decoupling layer are connected in parallel. The first gradient decoupling layer and the second gradient decoupling layer are used to decouple the parameter iteration process of the region generation network and the parameter iteration process of the feature extractor.

[0202] Figure 23 The model training apparatus of the illustrated embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0203] Figure 24 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure. The image processing apparatus provided in this embodiment can execute the processing flow provided in the image processing method embodiment, such as... Figure 24 As shown, the image processing apparatus 240 includes:

[0204] Acquisition module 241 is used to acquire the target image to be processed;

[0205] The input module 242 is used to input the target image into the trained machine learning model, so that the trained machine learning model outputs at least one target region in the target image and the confidence score of the category to which the target object in each target region belongs. The trained machine learning model is obtained according to the method described above.

[0206] The determination module 234 is used to determine the level of the target object in the target image based on the confidence level of the category to which the target object belongs in each target region.

[0207] Optionally, when determining the level of the target object in the target image based on the confidence level of the category to which the target object belongs in each target region, the determining module 234 is specifically used for:

[0208] Obtain a first image set and a second image set associated with the target image;

[0209] Based on the first image set and the second image set, the confidence level of the category to which the target object belongs in each target region is corrected to obtain the corrected confidence level;

[0210] Based on the corrected confidence level, the category of the target object in the target image is determined.

[0211] Figure 24 The image processing apparatus of the illustrated embodiment can be used to execute the technical solutions of the above method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0212] Figure 25 This is a schematic diagram of the scrap steel grading device provided in an embodiment of this disclosure. The scrap steel grading device provided in this embodiment can execute the processing flow provided in the scrap steel grading method embodiment, such as... Figure 25 As shown, the scrap steel grading device 250 includes:

[0213] Module 251 is used to acquire images of the whole vehicle of scrap steel at multiple levels during the unloading process;

[0214] Input module 252 is used to input the image of each level into the trained machine learning model, so that the trained machine learning model outputs at least one target region in the image of each level, and the category of the target object in each target region, wherein the trained machine learning model is obtained according to the method described above.

[0215] The determination module 253 is used to determine the grade of the whole vehicle scrap steel based on at least one target area in the image of each level and the category of the target object in each target area.

[0216] Optionally, the determining module 253 is further configured to: determine the proportion of impurities in the scrap steel of the whole vehicle based on at least one target region in the image of each level and the category of the target object in each target region.

[0217] Figure 25 The scrap steel grade determination device shown in the embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0218] The above describes the internal functions and structure of the model training device, image processing device, and scrap steel grade determination device, which can be implemented as an electronic device. Figure 26 A schematic diagram illustrating the structure of an electronic device embodiment provided in this disclosure. (See attached diagram.) Figure 26 As shown, the electronic device includes a memory 261 and a processor 262.

[0219] Memory 261 is used to store programs. In addition to the programs described above, memory 261 can also be configured to store various other data to support operation on the electronic device. Examples of such data include instructions for any application or method used to operate on the electronic device, contact data, phone book data, messages, images, videos, etc.

[0220] The memory 261 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0221] The processor 262 is coupled to the memory 261 and executes the program stored in the memory 261 for:

[0222] The model after the previous iteration training and the target dataset used in the previous iteration training are obtained. The target dataset includes at least a first dataset, which includes multiple first samples. The first sample includes a first image and text information of the target object in the first image. The first dataset is used for model training to obtain the model after the first iteration training. The model after the previous iteration training is the model after the first iteration training, or it is a model trained by at least one iteration training based on the model after the first iteration training.

[0223] The target dataset is updated using the model trained in the previous iteration.

[0224] Based on the updated target dataset, the model trained in the previous iteration is iteratively trained to obtain the model trained in the current iteration. After multiple iterations of training, a pre-trained model is obtained.

[0225] Furthermore, such as Figure 26 As shown, the electronic device may also include other components such as a communication component 263, a power supply component 264, an audio component 265, and a display 266. Figure 26 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 26 The components shown.

[0226] Communication component 263 is configured to facilitate wired or wireless communication between electronic devices and other devices. The electronic devices can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 263 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 263 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0227] Power supply component 264 provides power to various components of an electronic device. Power supply component 264 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0228] Audio component 265 is configured to output and / or input audio signals. For example, audio component 265 includes a microphone (MIC) configured to receive external audio signals when the electronic device is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 261 or transmitted via communication component 263. In some embodiments, audio component 265 also includes a speaker for outputting audio signals.

[0229] Display 266 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.

[0230] In addition, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the methods described in the above embodiments.

[0231] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0232] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A model training method, wherein, The method includes: The model after the previous iteration training and the target dataset used in the previous iteration training are obtained. The target dataset includes at least a first dataset, which includes multiple first samples. The first sample includes a first image and text information of the target object in the first image. The first dataset is used for model training to obtain the model after the first iteration training. The model after the previous iteration training is the model after the first iteration training, or it is a model trained by at least one iteration training based on the model after the first iteration training. The target dataset is updated using the model trained in the previous iteration. Based on the updated target dataset, the model trained in the previous iteration is iteratively trained to obtain the model trained in the current iteration. After multiple iterations of training, a pre-trained model is obtained. The method further includes: selecting a target subset from the unlabeled second dataset using the model trained in the previous iteration, wherein the evaluation metric of each second image in the target subset meets preset conditions, and the evaluation metric is determined by the first granularity score and the second granularity score of each second image; labeling each second image in the target subset using the model trained in the previous iteration to obtain a labeled target subset; updating the target dataset according to the labeled target subset, wherein the target dataset and the labeled target subset constitute the updated target dataset.

2. The method according to claim 1, wherein, Using the model trained in the previous iteration, a target subset is selected from the unlabeled second dataset, including: For each second image in the unlabeled second dataset, the second image is scored at a first granularity and a second granularity using the model trained in the previous iteration, to obtain the first granularity score and the second granularity score of the second image, where the first granularity is greater than the second granularity. Based on the first granularity score and the second granularity score corresponding to each second image, the evaluation index corresponding to each second image is determined. The second images in the second dataset that meet the preset conditions for evaluation metrics constitute the target subset.

3. The method according to claim 2, wherein, Using the model trained in the previous iteration, each second image in the target subset is labeled, including: For each second image in the target subset, at least one target region in the second image and the category of the target object in each target region are detected using the model trained in the previous iteration. The second image is labeled according to the category of the target object included in each target region in the second image.

4. A model training method, wherein, The method includes: Obtain the first and second images from the sample dataset; According to the machine learning model to be trained, the first image and the second image are detected respectively to obtain a first detection result of the first image and a second detection result of the second image. The initial parameters of the machine learning model to be trained are the parameters of the pre-trained model, which is obtained according to the method of any one of claims 1-3. Multiple first candidate regions in the first image and multiple second candidate regions in the second image are fused together to obtain multiple third candidate regions; Based on the plurality of third candidate regions, a third detection result is obtained, wherein the first detection result, the second detection result, and the third detection result each include at least one target region and the category of the target object in each target region; Based on the first detection result, the second detection result, and the third detection result, the machine learning model to be trained is trained to obtain the trained machine learning model.

5. The method according to claim 4, wherein, Based on the first detection result, the second detection result, and the third detection result, the machine learning model to be trained is trained to obtain a trained machine learning model, including: A loss function is constructed based on the difference between the annotation result of the first image and the first detection result, and the difference between the second detection result and the third detection result; The machine learning model to be trained is trained according to the loss function to obtain the trained machine learning model.

6. The method according to claim 4, wherein, Based on the first detection result, the second detection result, and the third detection result, the machine learning model to be trained is trained to obtain a trained machine learning model, including: A loss function is constructed based on the difference between the feature vector corresponding to the target region in the annotation result of the first image and the feature vector corresponding to the target region in the first detection result, as well as the difference between the feature vector corresponding to the target region in the second detection result and the feature vector corresponding to the target region in the third detection result. The machine learning model to be trained is trained according to the loss function to obtain the trained machine learning model.

7. The method according to claim 4, wherein, The machine learning model to be trained includes a first gradient decoupling layer, a second gradient decoupling layer, a region generation network, and a feature extractor. The first gradient decoupling layer is connected to the region generation network, and the second gradient decoupling layer is connected to the feature extractor. The first gradient decoupling layer and the second gradient decoupling layer are connected in parallel. The first gradient decoupling layer and the second gradient decoupling layer are used to decouple the parameter iteration process of the region generation network and the parameter iteration process of the feature extractor.

8. An image processing method, wherein, The method includes: Obtain the target image to be processed; The target image is input into a trained machine learning model, such that the trained machine learning model outputs at least one target region in the target image and the confidence level of the category to which the target object in each target region belongs, wherein the trained machine learning model is obtained according to the method of any one of claims 4-7; The level of the target object in the target image is determined based on the confidence level of the category to which the target object belongs in each target region.

9. The method according to claim 8, wherein, Based on the confidence level of the category to which the target object belongs in each target region, the level of the target object in the target image is determined, including: Obtain a first image set and a second image set associated with the target image; Based on the first image set and the second image set, the confidence level of the category to which the target object belongs in each target region is corrected to obtain the corrected confidence level; Based on the corrected confidence level, the category of the target object in the target image is determined.

10. A method for determining the grade of scrap steel, wherein, The method includes: Acquire images of the entire vehicle's scrap steel during the unloading process; The image at each level is input into the trained machine learning model, such that the trained machine learning model outputs at least one target region in the image at each level, and the category of the target object in each target region, wherein the trained machine learning model is obtained according to the method of any one of claims 4-7; The grade of the scrap steel from the vehicle is determined based on at least one target region in the image of each level and the category of the target object in each target region.

11. The method according to claim 10, wherein, The method further includes: The proportion of impurities in the scrap steel of the vehicle is determined based on at least one target region in the image of each level and the category of the target object in each target region.

12. A model training device, wherein, include: The acquisition module is used to acquire the model after the previous round of iteration training and the target dataset used in the previous round of iteration training. The target dataset includes at least a first dataset, which includes multiple first samples. The first sample includes a first image and text information of the target object in the first image. The first dataset is used for model training to obtain the model after the first round of iteration training. The model after the previous round of iteration training is the model after the first round of iteration training, or a model after at least one round of iteration training based on the model after the first round of iteration training. The update module is used to update the target dataset using the model trained in the previous iteration. The iterative training module is used to iteratively train the model after the previous round of iterative training based on the updated target dataset to obtain the model after the current round of iterative training. After multiple rounds of iterative training, a pre-trained model is obtained. The update module is further configured to: select a target subset from the unlabeled second dataset using the model trained in the previous iteration, wherein the evaluation metric of each second image in the target subset satisfies a preset condition, and the evaluation metric is determined by the first granularity score and the second granularity score of each second image; label each second image in the target subset using the model trained in the previous iteration to obtain a labeled target subset; and update the target dataset according to the labeled target subset, wherein the target dataset and the labeled target subset constitute the updated target dataset.

13. An electronic device, wherein, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-11.

Images