Training sample screening method, device, computer equipment and storage medium

Abstract

This application relates to a training sample selection method, apparatus, computer device, and storage medium. The method includes: randomly blocking neurons in an image processing model to obtain a blocking model, and performing image processing on a current training sample using the blocking model to obtain a first processing result; for the current training sample, determining a corresponding model metric representation vector based on the first processing results corresponding to multiple random blocking operations; obtaining a transformed sample obtained by transforming the current training sample, and performing image processing on the transformed sample using the image processing model to obtain a second processing result; determining a corresponding image metric representation vector based on the second processing results of multiple corresponding transformed samples; combining the model metric representation vector and the image metric representation vector to obtain a comprehensive metric result; and selecting target training samples based on the comprehensive metric result of each training sample in the training sample set. This method can improve the training effect of the model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a training sample screening method, apparatus, computer equipment and storage medium. Background Art

[0002] With the development of science and technology, the application of image processing models is becoming more and more extensive. For example, image processing models can be used for face recognition, heat map key point recognition, or license plate recognition.

[0003] Before processing an image through an image processing model, the image processing model needs to be trained. Currently, the main method is to randomly select training samples, label the randomly selected training samples, and then train the image processing model by adding the labeled training samples.

[0004] However, when there are many similar and redundant training samples in the training sample library, the method of randomly selecting training samples is likely to select many worthless training samples, thereby reducing the training effect of the image processing model based on worthless training samples. Summary of the invention

[0005] Based on this, it is necessary to provide a training sample screening method, device, computer equipment and storage medium that can improve the training effect of the image recognition model in response to the above technical problems.

[0006] A training sample screening method, the method comprising:

[0007] Randomly blocking neurons in the image processing model to obtain a blocking model, and performing image processing on a current training sample in a training sample set by using the blocking model to obtain a first processing result;

[0008] For the current training sample, based on first processing results corresponding to the multiple random blockings respectively, determining a corresponding model metric characterization vector, wherein the model metric characterization vector is used to characterize model uncertainty;

[0009] Acquire a transformed sample obtained by performing image transformation on the current training sample, and perform image processing on the transformed sample through the image processing model to obtain a second processing result;

[0010] For the current training sample, based on the second processing results of the corresponding plurality of transformed samples, determining a corresponding image metric representation vector, wherein the image metric representation vector is used to represent image uncertainty;

[0011] The model metric representation vector and the image metric representation vector are integrated to obtain a comprehensive metric result, and a target training sample is screened out according to the comprehensive metric result of each training sample in the training sample set; the screened target training sample is used to train the image processing model.

[0012] In one embodiment, performing image processing on the transformed sample by using the image processing model to obtain a second processing result includes:

[0013] The image processing model is used to perform image processing on each transformed sample corresponding to the current training sample to obtain a plurality of second processing results corresponding to the current training sample.

[0014] A training sample screening device, the device comprising:

[0015] A model uncertainty acquisition module is used to randomly block neurons in the image processing model to obtain a blocking model, and perform image processing on a current training sample in a training sample set through the blocking model to obtain a first processing result; for the current training sample, based on the first processing results corresponding to multiple random blockages, a corresponding model metric representation vector is determined, and the model metric representation vector is used to represent model uncertainty;

[0016] an image uncertainty acquisition module, configured to acquire a transformed sample obtained by performing image transformation on the current training sample, and perform image processing on the transformed sample through the image processing model to obtain a second processing result; for the current training sample, based on the second processing results of the corresponding plurality of transformed samples, determine a corresponding image metric representation vector, wherein the image metric representation vector is used to represent image uncertainty;

[0017] A screening module is used to synthesize the model metric representation vector and the image metric representation vector to obtain a comprehensive metric result, and screen out target training samples according to the comprehensive metric result of each training sample in the training sample set; the screened target training samples are used to train the image processing model.

[0018] In one embodiment, the model uncertainty acquisition module also includes a first processing result acquisition module, which is used to randomly block a first preset number of neurons in the image recognition model for the current training sample in the training sample set to obtain a blocking model, and perform image recognition on the current training sample through the blocking model to obtain a first processing result; enter the next round of image recognition, return to the step of randomly blocking the first preset number of neurons in the image recognition model and continue to execute until a first preset stop condition is met, and obtain multiple first processing results corresponding to the current training sample.

[0019] In one embodiment, the model uncertainty acquisition module is also used to perform an averaging operation on multiple first processing results corresponding to multiple random blockages for the current training sample to obtain a first mean result corresponding to the current training sample; determine the difference between each first processing result corresponding to the current training sample and the first mean result to obtain a first difference result corresponding to each first processing result; and combine multiple first difference results to obtain a model metric representation vector of the current training sample.

[0020] In one embodiment, the model uncertainty acquisition module is also used to perform a mean operation on multiple first difference results to obtain a first mean difference; perform a variance operation on multiple first difference results to obtain a first variance difference; determine the maximum value among multiple first difference results to obtain a first maximum difference; and combine the first mean difference, the first variance difference and the first maximum difference to obtain a model metric representation vector of the current training sample.

[0021] In one embodiment, the image uncertainty acquisition module also includes an image transformation module, which is used to adjust at least one of the brightness, contrast, saturation and sharpness of the current training sample to obtain a color perturbation sample corresponding to the current training sample; perform pixel displacement on the pixel points in the color perturbation sample to obtain a transformed sample corresponding to the current training sample; enter the next round of image transformation, return to the step of adjusting at least one of the brightness, contrast, saturation and sharpness of the current training sample and continue to execute until a second preset stop condition is met, and obtain multiple transformed samples corresponding to the current training sample.

[0022] In one embodiment, the image uncertainty acquisition module is also used to perform an averaging operation on multiple second processing results corresponding to the current training sample to obtain a second mean result corresponding to the current training sample; determine the difference between each second processing result corresponding to the current training sample and the second mean result to obtain a second difference result corresponding to each second processing result; and combine multiple second difference results to obtain an image metric representation vector corresponding to the current training sample.

[0023] In one embodiment, the image uncertainty acquisition module is also used to perform an averaging operation on multiple second difference results to obtain a second mean difference; perform a variance operation on multiple second difference results to obtain a second variance difference; determine the maximum value among the multiple second difference results to obtain a second maximum difference; and combine the second mean difference, the second variance difference and the second maximum difference to obtain the image metric representation vector of the current training sample.

[0024] In one embodiment, the training sample set includes a first training sample set and a second training sample set, the first training sample set includes a first training sample with a label; the second training sample set includes a second training sample without a label; the screening module is also used to splice the model metric representation vector and the image metric representation vector corresponding to each of the first training samples, respectively, to obtain the first comprehensive measurement result corresponding to each of the first training samples; and to splice the model metric representation vector and the image metric representation vector corresponding to each of the second training samples, respectively, to obtain the second comprehensive measurement result corresponding to each of the second training samples.

[0025] In one embodiment, the screening module 806 is also used to

[0026] For the first training sample set and the second training sample set corresponding to the current screening, respectively determine the difference between the first comprehensive metric result of each first training sample in the first training sample set and the second comprehensive metric result of each second training sample in the second training sample set, and obtain multiple metric result differences corresponding to each second comprehensive metric result; according to the multiple metric result differences corresponding to each second comprehensive metric result, screen the target training sample of the current time from the second training sample set; label the target training sample of the current time, store the labeled target training sample in the first training sample set, obtain an updated first training sample set, and delete the target training sample in the second training sample set, and obtain an updated second training sample set; use the updated first training sample set as the first training sample set corresponding to the next screening, use the updated second training sample set as the second training sample set corresponding to the next screening, enter the next screening and return to the step of respectively determining the difference between the first comprehensive metric result of each first training sample in the first training sample set and the second comprehensive metric result of each second training sample in the second training sample set, and obtain multiple metric result differences corresponding to each second comprehensive metric result, and continue to execute until the third stop condition is reached, and the second preset number of target training samples is obtained.

[0027] In one embodiment, the screening module also includes a mapping module, which is used to normalize the first comprehensive measurement result to obtain a first normalized result, and map the first normalized result through a mapping function to obtain a first mapping result; normalize the second comprehensive measurement result to obtain a second normalized result, and map the second normalized result through a mapping function to obtain a second mapping result; respectively determine the difference between the first mapping result corresponding to each first training sample in the first training sample set and the second mapping result corresponding to each second training sample in the second training sample set, and obtain multiple measurement result differences corresponding to each of the second comprehensive measurement results.

[0028] In one embodiment, the screening module also includes a target determination module, which is used to use the minimum measurement result difference among the multiple measurement result differences corresponding to each of the second comprehensive measurement results as the target measurement result difference of the corresponding second comprehensive measurement result; screen out the maximum target measurement result difference from the multiple target measurement result differences, and use the second comprehensive measurement result with the maximum target measurement result difference as the target second comprehensive measurement result; and use the second training sample with the target second comprehensive measurement result as the target training sample.

[0029] In one embodiment, the training sample set includes a first training sample set, the first training sample set includes a first training sample with a labeled label, the training sample screening device is also used to label the target training sample, and store the labeled target training sample in the first training sample set to obtain an updated first training sample set; the image processing model is trained by the updated first training sample set until a fourth preset stop condition is reached, to obtain a trained image processing model; the trained image processing model is used to identify the image to be identified.

[0030] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0031] Randomly blocking neurons in the image processing model to obtain a blocking model, and performing image processing on a current training sample in a training sample set by using the blocking model to obtain a first processing result;

[0032] For the current training sample, based on first processing results corresponding to the multiple random blockings respectively, determining a corresponding model metric characterization vector, wherein the model metric characterization vector is used to characterize model uncertainty;

[0033] Acquire a transformed sample obtained by performing image transformation on the current training sample, and perform image processing on the transformed sample through the image processing model to obtain a second processing result;

[0034] For the current training sample, based on the second processing results of the corresponding plurality of transformed samples, determining a corresponding image metric representation vector, wherein the image metric representation vector is used to represent image uncertainty;

[0035] The model metric representation vector and the image metric representation vector are integrated to obtain a comprehensive metric result, and a target training sample is screened out according to the comprehensive metric result of each training sample in the training sample set; the screened target training sample is used to train the image processing model.

[0036] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0037] Randomly blocking neurons in the image processing model to obtain a blocking model, and performing image processing on a current training sample in a training sample set by using the blocking model to obtain a first processing result;

[0038] For the current training sample, based on first processing results corresponding to the multiple random blockings respectively, determining a corresponding model metric characterization vector, wherein the model metric characterization vector is used to characterize model uncertainty;

[0039] Acquire a transformed sample obtained by performing image transformation on the current training sample, and perform image processing on the transformed sample through the image processing model to obtain a second processing result;

[0040] For the current training sample, based on the second processing results of the corresponding plurality of transformed samples, determining a corresponding image metric representation vector, wherein the image metric representation vector is used to represent image uncertainty;

[0041] The model metric representation vector and the image metric representation vector are integrated to obtain a comprehensive metric result, and a target training sample is screened out according to the comprehensive metric result of each training sample in the training sample set; the screened target training sample is used to train the image processing model.

[0042] A computer program product or a computer program includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the following steps: randomly blocking neurons in an image processing model to obtain a blocking model, and performing image processing on a current training sample in a training sample set through the blocking model to obtain a first processing result; for the current training sample, based on the first processing results corresponding to multiple random blockages, respectively, determining a corresponding model metric characterization vector, the model metric characterization vector is used to characterize model uncertainty; obtaining a transformed sample obtained by performing image transformation on the current training sample, and performing image processing on the transformed sample through the image processing model to obtain a second processing result; for the current training sample, based on the second processing results of the corresponding multiple transformed samples, determining a corresponding image metric characterization vector, the image metric characterization vector is used to characterize image uncertainty; synthesizing the model metric characterization vector and the image metric characterization vector to obtain a comprehensive metric result, and screening out a target training sample according to the comprehensive metric result of each training sample in the training sample set; the screened target training sample is used to train the image processing model.

[0043] The above-mentioned training sample screening method, device, computer equipment, storage medium and computer program can obtain a blocking model by randomly blocking neurons in the image processing model, thereby performing image processing on the current training sample in the sample training set through the blocking model to obtain a first processing result, and obtain a model metric characterization vector corresponding to the current training sample for characterizing model uncertainty based on the first processing result. By performing image transformation on the current training sample in the sample training set, a transformed sample can be obtained, thereby performing image processing on the transformed sample through the image processing model to obtain a second processing result, and obtain an image metric characterization vector corresponding to the current training sample for characterizing image uncertainty based on the second processing result. By obtaining the model metric characterization vector and the image metric characterization vector corresponding to each training sample in the sample training set, a comprehensive metric result for characterizing the difficulty of the training sample can be obtained based on the model metric characterization vector and the image metric characterization vector, so that the target training sample that is more conducive to model training can be screened out from the training sample set based on the comprehensive metric result. Since the present application can screen out target training samples through the comprehensive measurement results used to characterize the difficulty of the samples, compared with the traditional random screening of training samples, the present application can screen out valuable training samples, thereby improving the training effect of the image processing model based on the valuable training samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1A diagram showing an application environment of a training sample screening method in an embodiment;

[0045] Figure 2 A schematic diagram of a flow chart of a training sample screening method in one embodiment;

[0046] Figure 3 is a schematic diagram of a blocking model in one embodiment;

[0047] Figure 4 is a schematic diagram of neuron blocking in one embodiment;

[0048] Figure 5 A schematic diagram of a framework for iterative model training in one embodiment;

[0049] Figure 6 is a schematic flow chart of a method for screening target training samples in one embodiment;

[0050] Figure 7 is a flow chart of a training sample screening method in a specific embodiment;

[0051] Figure 8 is a structural block diagram of a training sample screening device in one embodiment;

[0052] Fig. 9 is a structural block diagram of a training sample screening device in another embodiment;

[0053] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] Figure 1 FIG. 1 is an application environment diagram of a training sample screening method in an embodiment. Figure 1 , the training sample screening method is applied to a training sample screening system 100. The training sample screening system 100 includes a terminal 102 and a server 104. When the terminal 102 obtains a training sample, the terminal 102 may send the training sample to the server 104, so that the server 104 determines a comprehensive metric result corresponding to the training sample, and screens out a target training sample from a training sample library through the comprehensive metric result corresponding to each training sample, so as to train the image processing model through the screened target training sample.

[0056] The server 104 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, a smart TV, etc., but is not limited thereto. The terminal 102 and the server 104 may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0057] The present application relates to the field of artificial intelligence (AI), for example, the comprehensive measurement results of training samples can be determined by artificial intelligence. Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0058] It is worth noting that the present application also relates to the field of blockchain technology, such as the training sample screening method disclosed in the present application, wherein the screened training samples can be stored on the blockchain.

[0059] The present application also relates to the field of map-vehicle networking. For example, the terminal in the present application may be a vehicle-mounted terminal in which an image processing model is deployed. Through the above-mentioned training sample screening method, target training samples can be screened out from the training sample set, and the image processing model in the vehicle-mounted terminal can be trained through the target training samples to process the image based on the trained image processing model, for example, to identify the driving environment map based on the trained image processing model, so as to navigate based on the recognition result.

[0060] In order to better understand the training sample screening method in the embodiment of the present application, the overall application scenario of the present application is introduced below:

[0061] For active learning, currently, the main method is to randomly select target training samples from the training sample set, and then label the randomly selected target training samples, and train the image processing model through the labeled target training samples. However, the method of randomly selecting target training samples will lead to poor training effect of the model. Therefore, in order to select target training samples that are more conducive to model training from the training sample set, the computer device can determine the potential uncertainty of the training sample for the current model reasoning output and the difficulty of the training sample, and select the target training sample from the training sample set based on the potential uncertainty of the training sample for the current model reasoning output and the difficulty of the training sample. Among them, when selecting training samples, if only training samples with large uncertainty are selected, or only difficult training samples are selected, it will make the image processing model difficult to train and cannot achieve the best effect. Therefore, the computer device can comprehensively consider the potential uncertainty of the training sample for the current model reasoning output and the difficulty of the training sample according to the needs, and select the target training sample that is conducive to model training from the training sample set.

[0062] In one embodiment, Figure 2 As shown, a training sample screening method is provided, which is applied to Figure 1 The computer device in the example is used for explanation, wherein the computer device includes a terminal 102 and a server 104, and the training sample screening method includes the following steps:

[0063] Step S202: randomly block neurons in the image processing model to obtain a blocking model, and perform image processing on the current training sample in the training sample set by using the blocking model to obtain a first processing result.

[0064] Among them, neurons refer to the mathematical model in the machine learning model. For a neuron, it receives information transmitted by other neurons, processes the received information, obtains the processing results, and transmits the processing results to other neurons. Blocking refers to pausing the work of neurons, so that they stop receiving information transmitted by other neurons and stop processing the information.

[0065] Image processing refers to fully mining the information fed back by the image through a series of algorithms, wherein image processing can specifically be image recognition, image decomposition, etc. For example, the image processing can be for recognizing facial images, recognizing key points in heat maps, recognizing driving environment scene maps, etc. In one embodiment, the image can be processed by an image processing model, wherein the image processing model can specifically be a machine learning model obtained based on active learning. Active learning refers to selecting a part of the training samples from the training sample set and providing them to the evaluators for subjective scoring. The evaluators subjectively score the selected training samples to obtain training samples labeled with subjective scores, thereby training the machine learning model through the training samples labeled with subjective scores. Active learning can actively select some training samples for model training through a specific selection strategy, thereby effectively reducing the amount of samples that need to be labeled while ensuring model performance.

[0066] Specifically, when it is necessary to determine the uncertainty of the output result obtained after the image processing model infers the training sample, that is, when it is necessary to determine the model metric representation vector used to characterize the uncertainty of the model, the computer device can generate a blocked image processing model corresponding to the image processing model, and perform image processing on each training sample in the training sample set through the blocked image processing model to obtain the processing result corresponding to each training sample, and use the processing result as the first processing result. In order to better understand this embodiment, the following is an example of the current training sample in the training sample set, wherein the current training sample is any training sample in the training sample set. When the current training sample set is obtained, the computer device can perform multiple image processing on the current training sample, wherein for each image processing, the neurons in the image processing model are randomly blocked to obtain a blocking model, and the current training sample is image processed through the blocking model to obtain a first processing result.

[0067] In one implementation, reference Figure 3 , a neuron blocking layer can be added after the fully connected layer of the image processing model, and some neurons in the image processing model can be randomly blocked by the neuron blocking layer to obtain a blocking model. Among them, the computer device can determine the probability value of the neuron blocking layer randomly blocking the neurons, and thus determine the number of randomly blocked neurons based on the determined probability value. Since the image processing model can be stacked by multiple basic modules with similar structures, a neuron blocking layer can be added after each basic module, and the inserted neuron blocking layer can be placed in the inference mode, so as to randomly block some neurons in the corresponding basic model based on the neuron blocking layer in the inference mode. Figure 3 A schematic diagram of a blocking model in one embodiment is shown.

[0068] In one embodiment, the image processing model can be pre-trained to obtain a pre-trained image processing model, and a neuron blocking layer can be added to the pre-trained image processing model, that is, a neuron blocking layer can be added to the image processing model in the testing phase to obtain a blocking model. Since, for the latest image processing model, adding a neuron blocking layer may lead to a decrease in the performance of the model itself, the probability of a decrease in model performance can be reduced by using a neuron blocking layer in the testing phase and a low probability of neuron blocking.

[0069] In one embodiment, reference Figure 4 , Figure 4 FIG. 2 shows a schematic diagram of neuron blocking in one embodiment. Figure 4 As shown, the image processing model may be composed of multiple neurons at different levels. When the training sample is input into the image processing model, the neuron blocking layer in the image processing model will randomly block some neurons in the image processing model according to the neuron blocking probability to obtain a blocking model, and the input training sample is processed by the blocking model to obtain the first processing result of the training sample. For example, when the image processing model is a machine learning model for recognizing faces in an image, the computer device may recognize faces in the training sample through the blocking model corresponding to the image processing model to obtain face recognition results, and use the face recognition results as the first processing result; for another example, when the image processing model is a machine learning model for recognizing key points of a heat map, the computer device may recognize key points of the heat map through the blocking model corresponding to the image processing model to obtain a key point recognition model, and use the key point recognition model as the first processing result.

[0070] In one embodiment, neurons in an image processing model are randomly blocked to obtain a blocking model, and image processing is performed on a current training sample in a training sample set through the blocking model to obtain a first processing result, including: for the current training sample in the training sample set, a first preset number of neurons in an image recognition model are randomly blocked to obtain a blocking model, and image recognition is performed on the current training sample through the blocking model to obtain a first processing result; entering the next round of image recognition, returning to the step of randomly blocking the first preset number of neurons in the image recognition model and continuing to execute until a first preset stop condition is met, and multiple first processing results corresponding to the current training sample are obtained.

[0071] Specifically, for the current training sample in the training sample set, the computer device may perform multiple image processing on the current training sample, wherein, during each image processing, a first preset number of neurons in the image processing model are randomly blocked to obtain a blocking model, and the current training sample is image processed by the blocking model to obtain a first processing result. For example, for training sample A, the computer device may input training sample A into the image processing model, and randomly block some neurons in the image processing model in the forward process, and perform image processing on training sample A through unblocked neurons in the image processing model to obtain the first first processing result of training sample A. The computer device then inputs training sample A into the image processing model, and randomly blocks some neurons in the image processing model in the forward process, and performs image processing on training sample A through unblocked neurons in the image processing model to obtain the second first processing result of training sample A. The iteration is repeated until the first preset stop condition is met, and the first preset number of first processing results corresponding to training sample A are obtained.

[0072] It is easy to understand that since the neurons in the image recognition model are randomly blocked, the blocking model for each image recognition may be inconsistent. As a result, the first processing result obtained each time may be inconsistent, and thus a model metric representation vector for reflecting model uncertainty is obtained through multiple inconsistent first processing results.

[0073] In this embodiment, by performing multiple image recognition on the current training samples, multiple first processing results corresponding to the current training samples can be obtained, so that the model metric representation vector for reflecting the model uncertainty can be quickly obtained through the multiple first processing results.

[0074] Step S204: for the current training sample, based on the first processing results corresponding to the multiple random blockings, a corresponding model metric characterization vector is determined, where the model metric characterization vector is used to characterize the model uncertainty.

[0075] Specifically, since the training sample is processed by a blocking model that randomly blocks neurons to obtain a first processing result, the uncertainty of the model can be obtained through the first processing result. Among them, model uncertainty refers to the cognitive uncertainty caused by the uncertainty of neurons involved in data processing in the model. When it is necessary to determine the model metric representation vector of the training sample, for each training sample in the training sample set, the computer device can determine multiple first processing results corresponding to the current training sample, and process the multiple first processing results corresponding to the current training sample to obtain the model metric representation vector corresponding to the current training sample. Among them, the model metric representation vector is used to characterize the model uncertainty.

[0076] In one embodiment, for the current training sample, based on the first processing results corresponding to multiple random blockages, a corresponding model metric representation vector is determined, including: for the current training sample, averaging multiple first processing results corresponding to multiple random blockages to obtain a first mean result corresponding to the current training sample; determining the difference between each first processing result corresponding to the current training sample and the first mean result to obtain a first difference result corresponding to each first processing result; and combining multiple first difference results to obtain the model metric representation vector of the current training sample.

[0077] Specifically, when multiple first processing results corresponding to the current training sample in the training sample set are obtained, the computer device can obtain the model metric characterization vector corresponding to the current training sample based on the multiple first processing results corresponding to the current training sample. It is easy to understand that the current training sample can be any training sample in the training sample set. When it is necessary to determine the model metric characterization vector of the current training sample, the computer device can obtain multiple first processing results of the current training sample, and perform an average operation on the multiple first processing results corresponding to the current training sample to obtain the first mean result corresponding to the current training sample. For example, the computer device can determine the number of multiple first processing results, and sum the multiple first processing results to obtain the sum of the first processing results, and divide the sum of the first processing results by the number of multiple first processing results to obtain the first mean result.

[0078] Further, for each of the multiple first processing results corresponding to the current training sample, the computer device may respectively determine the difference between each first processing result and the first mean result, thereby obtaining a first difference result corresponding to each first processing result. The computer device may synthesize the multiple first difference results corresponding to the current training sample to obtain a model metric representation vector corresponding to the current training sample.

[0079] It is easy to understand that when the computer device obtains multiple first processing results corresponding to the current training sample, it can immediately process the multiple first processing results corresponding to the current training sample to obtain the model metric representation vector corresponding to the current training sample. The computer device can also process the multiple first processing results corresponding to each training sample in the training sample set to obtain the model metric representation vector corresponding to each training sample.

[0080] In one embodiment, for different image processing tasks, the first difference result can be determined in different ways. For example, for a coordinate regression task, the Manhattan distance or Euclidean distance between the first processing result and the first mean result can be determined, and the Manhattan distance or Euclidean distance can be used as the first difference result; for a human key point recognition task, the first difference result can be determined by determining the similarity between the key point pointed to by the first processing result and the key point pointed to by the first mean.

[0081] In one embodiment, the first processing result can be determined by the formula O=M(I), where I is the training sample, M is the blocking model, and O is the first processing result output by the blocking model. The multiple first difference results corresponding to the current training sample can be determined by the following formula: Among them, i represents the i-th round of image recognition, O i represents the first processing result obtained by the i-th round of image recognition, represents the first mean value result, and f represents the metric function used to determine the difference between the first processing result and the first mean value result. When the image processing task is a human key point recognition task, the metric function can be Among them, OKS represents the similarity between the key point pointed to by the first processing result and the key point pointed to by the first mean.

[0082] In one embodiment, the first difference result can be determined by the result indirectly output by the model. For example, the supervised loss function of the intermediate feature layer of the image processing model can be used as a metric function, so that the result output by the supervised loss function is used as the first processing result, and the mean of multiple first processing results is used as the first mean result. Since the metric function is mainly used to express the difference between the data after model inference and the mean of multiple inferences, the input of the metric function does not necessarily need to be the final output result of the model, but can also be an intermediate result of the output.

[0083] In the above embodiment, it is only necessary to determine the first difference result between each first processing result and the first mean result, and then obtain the model metric characterization vector based on the first difference result, thereby greatly improving the efficiency of determining the model metric characterization vector.

[0084] In one embodiment, a plurality of first difference results are integrated to obtain a model metric characterization vector of the current training sample, including: performing a mean operation on the plurality of first difference results to obtain a first mean difference; performing a variance operation on the plurality of first difference results to obtain a first variance difference; determining the maximum value among the plurality of first difference results to obtain a first maximum difference; and integrating the first mean difference, the first variance difference and the first maximum difference to obtain the model metric characterization vector of the current training sample.

[0085] Specifically, when a plurality of first difference results corresponding to the current training sample are obtained, the computer device may perform a mean operation on the plurality of first difference results to obtain a first mean difference. For example, the computer device may determine the number of the plurality of first difference results, and perform a summation process on the plurality of first difference results to obtain the sum of the first difference results, and divide the sum of the first difference results by the number of the plurality of first difference results to obtain the first mean difference. Further, the computer device may perform a variance operation on the plurality of first difference results to obtain a first variance difference. For example, the computer device may obtain the first variance difference by determining the difference between the first difference result and the first mean difference. Further, the computer device determines the maximum value among the plurality of first difference results to obtain a first maximum difference, and combines the first mean difference, the first variance difference and the first maximum difference to obtain a model metric representation vector of the current training sample.

[0086] It is easy to understand that for each training sample in the training sample set, the computer device can determine a set of first difference results LModel corresponding to each training sample. 1,…,T , where LModel i Represents the first difference result obtained by the training sample in the i-th round of image recognition. Therefore, the computer device can determine the corresponding first mean difference, first variance difference and first maximum difference for each training sample in the training sample set, and then determine the model metric representation vector corresponding to each training sample based on the first mean difference, first variance difference and first maximum difference.

[0087] In one embodiment, the computer device may determine the model metric representation vector by the following formula: in, Represents the model metric representation vector, mean(LModel) represents the first mean difference, std(LModel) represents the first variance difference, and max(LModel) represents the first maximum difference.

[0088] In the above embodiment, since the model metric characterization vector includes the metric average value, metric fluctuation degree and maximum fluctuation range information introduced due to model uncertainty, the subsequent comprehensive metric result determined based on multiple information can be more accurate and the information contained can be richer.

[0089] Step S206, obtaining a transformed sample obtained by performing image transformation on the current training sample, and performing image processing on the transformed sample through the image processing model to obtain a second processing result.

[0090] Specifically, in order to obtain an image metric representation vector for characterizing image uncertainty, the computer device may perform an image transformation on the current training sample in the training sample set to obtain a transformed sample, and input the transformed sample into an image processing model, and perform image processing on the transformed sample through the image processing model to obtain a second processing result corresponding to the transformed sample. Since the transformed sample is a sample obtained by performing an image change on the current training sample, there is a corresponding relationship between the transformed sample and the current training sample. Therefore, the second processing result corresponding to the transformed sample is also the second processing result corresponding to the current training sample.

[0091] In one embodiment, performing image processing on the transformed sample through an image processing model to obtain a second processing result includes: performing image processing on each transformed sample corresponding to the current training sample through the image processing model to obtain multiple second processing results corresponding to the current training sample.

[0092] Specifically, since the image processing model extracts semantic features by convolving image pixels, for the same image, perturbing pixels in the image or performing pixel displacement will change the semantic features extracted by the model, thereby causing the image processing model to output different or even erroneous results. Therefore, in order to obtain an image metric representation vector for characterizing model uncertainty, the computer device may perform multiple image transformations on each training sample in the training sample set, that is, perform multiple pixel perturbations or pixel displacements on each training sample set to obtain multiple transformed samples corresponding to each training sample, and perform image processing on each transformed sample through the image processing model to obtain a second processing result corresponding to each transformed sample. Further, the computer device obtains multiple second processing results corresponding to each training sample based on the second processing results corresponding to each transformed sample, thereby obtaining an image metric representation vector for characterizing image uncertainty based on the multiple second processing results corresponding to each training sample. Image uncertainty refers to cognitive uncertainty caused by the uncertainty of the image input to the model.

[0093] In order to better understand the present embodiment, the following description is made by taking the current training sample in the training sample set as an example, wherein the current training sample can be any training sample in the training sample set. When the current training sample is obtained, the computer device can perform image transformation on the current training sample to obtain a transformed sample of the first transformation, and input the transformed sample of the first transformation into the image processing model to obtain a first second processing result. The computer device can perform image transformation on the current training sample again to obtain a transformed sample of the second transformation, and input the transformed sample of the second transformation into the image processing model to obtain a second second processing result. Iterate in sequence until the second preset stop condition is met, and obtain multiple second processing results corresponding to the current training sample.

[0094] It is easy to understand that the computer device can also perform multiple image transformations on the current training sample to obtain multiple transformed images corresponding to the current training sample, and then input the multiple transformed images into the image processing model to obtain multiple second processing results corresponding to the current training sample.

[0095] In this embodiment, since the transformed image is obtained by performing image transformation on the training sample, the uncertainty of the image can be obtained by performing image processing on the transformed image, so that the difficulty of the sample image can be determined subsequently by the image uncertainty, and the target sample image can be determined by the difficulty of the sample image.

[0096] In one embodiment, the above method also includes a step of generating a transformed sample, and the step of generating the transformed sample includes: adjusting at least one of the brightness, contrast, saturation and sharpness of the current training sample to obtain a color perturbation sample corresponding to the current training sample; performing pixel displacement on the pixel points in the color perturbation sample to obtain a transformed sample corresponding to the current training sample; entering the next round of image transformation, returning to the step of adjusting at least one of the brightness, contrast, saturation and sharpness of the current training sample and continuing to execute until a second preset stop condition is met, and obtaining multiple transformed samples corresponding to the current training sample.

[0097] Specifically, the computer device may perform pixel displacement and color perturbation on each training sample in the sample training set to obtain a transformed sample. For example, when the color perturbation of the current training sample is required, the computer device may adjust at least one of the brightness, contrast, saturation and sharpness of the current training sample to obtain a color perturbation sample corresponding to the current training sample. The current training sample is any training sample in the training sample set. The adjustment range of the random adjustment may be pre-set so that the computer device may adjust the brightness, contrast, saturation and sharpness within the preset adjustment range. The computer device may adjust the brightness, contrast, saturation and sharpness of the current training sample, and the computer device may also adjust one of the brightness, contrast, saturation and sharpness of the current training sample.

[0098] Furthermore, when a color perturbation sample corresponding to the current training sample is obtained, the computer device may randomly shift the pixels in the color perturbation sample to obtain a transformed sample corresponding to the current training sample. It is easy to understand that the computer device may first perform color perturbation on the current training sample and then perform pixel shifting to obtain a transformed sample; it may first perform pixel shifting on the current training sample and then perform color perturbation to obtain a transformed sample; it may also only perform color perturbation or pixel shifting on the current training sample to obtain a transformed sample. This embodiment is not limited here.

[0099] Furthermore, the computer enters the next round of image transformation, continues to perform color perturbation and pixel displacement on the current training sample in the training sample set, and obtains the next round of transformation samples. This iteration is stopped when the second preset stop condition is met, and multiple transformation samples corresponding to the current training sample are obtained.

[0100] In one embodiment, pixel displacement can be performed by affine transformation. For example, the affine transformation matrix can be used to randomly move and scale some pixels in the color perturbation sample to obtain the corresponding transformed sample. Furthermore, when the transformed sample is obtained, the transformed sample can be input into the image processing model, and the corresponding processing result is output by the image processing model. Since scaling displacement is introduced in the transformed sample, for image processing tasks such as segmentation and key points that output pixel information, it is also necessary to perform a corresponding inverse transformation on the processing result output by the image processing model to obtain a second processing result aligned to the original training sample.

[0101] In one embodiment, the computer device may determine a plurality of transformed samples corresponding to the current training sample by the following formula: I′ i =S i (I), i = 1, ..., T, where I represents the current training sample, I' irepresents the i-th transformed sample corresponding to the training sample I. The computer device can determine the processing result output by the image processing model through the following formula: U′ i =M(I′ i ), where M is the image processing model, U′ i Based on the transformed sample I′ i Since the scaling displacement is introduced in the transformed sample, the computer device can perform an inverse transformation on the processing result output by the image processing model through the following formula to obtain the second processing result Among them, U′ i is the processing result output by the image processing model. is the inverse transformation of the affine transformation, U i is the second processing result.

[0102] In the above embodiment, by randomly performing color perturbation and pixel displacement, a transformation sample containing image uncertainty can be obtained, and then an image metric representation vector can be obtained through the transformation sample containing image uncertainty.

[0103] Step S208: for the current training sample, based on the second processing results of the corresponding plurality of transformed samples, determine a corresponding image metric characterization vector, where the image metric characterization vector is used to characterize image uncertainty.

[0104] Specifically, the computer device may obtain an image metric representation vector of the current training sample according to the multiple second processing results corresponding to the current training sample. The image metric representation vector is used to represent image uncertainty. In one embodiment, the longer the modulus of the image metric representation vector is, the more difficult the training sample is, and thus the more difficult it is to perform model training based on the training sample.

[0105] In one embodiment, for each training sample in the training sample set, the computer device can immediately process the multiple second processing results corresponding to the current training sample when obtaining the multiple second processing results corresponding to the current training sample, and obtain the image metric representation vector corresponding to the current training sample. The computer device can also process the multiple second processing results corresponding to each training sample in the training sample set when obtaining the multiple second processing results corresponding to each training sample, and obtain the image metric representation vector corresponding to each training sample. This embodiment is not limited here.

[0106] In one embodiment, for the current training sample, the corresponding image metric representation vector is determined based on the second processing results of the corresponding multiple transformed samples, including: for the current training sample, performing an averaging operation on the multiple second processing results corresponding to the current training sample to obtain a second mean result corresponding to the current training sample; determining the difference between each second processing result corresponding to the current training sample and the second mean result to obtain a second difference result corresponding to each second processing result; and combining the multiple second difference results to obtain an image metric representation vector corresponding to the current training sample.

[0107] Specifically, when multiple second processing results corresponding to the current training sample are obtained, the computer device may perform an average operation on the multiple second processing results to obtain a second mean result corresponding to the current training sample. For example, the computer device may determine the number of multiple second processing results, and superimpose the multiple second processing results to obtain the sum of the second processing results, and divide the sum of the sum of the second processing results by the number of the multiple second processing results to obtain the second mean result. Furthermore, for each second processing result of the multiple second processing results corresponding to the current training sample, the computer device respectively determines the difference between each second processing result and the second mean result to obtain a second difference result corresponding to each second processing result. Thus, the computer device can combine the multiple second difference results to obtain an image metric representation vector corresponding to the current training sample.

[0108] In one embodiment, for different image processing tasks, the second difference result can be determined in different ways. For example, for a coordinate regression task, the Manhattan distance or Euclidean distance between the second processing result and the second mean result can be determined, and the Manhattan distance or Euclidean distance can be used as the second difference result; for a human key point recognition task, the second difference result can be determined by determining the similarity between the key point pointed to by the second processing result and the key point pointed to by the second mean.

[0109] In one embodiment, the second difference result may be determined by the following formula: Among them, i represents the i-th round of image recognition, U i represents the second processing result obtained by the i-th round of image recognition, represents the second mean value result, and f represents the metric function used to determine the difference between the second processing result and the second mean value result. When the image processing task is a human key point recognition task, the metric function can be Among them, OKS represents the similarity between the key point pointed to by the second processing result and the key point pointed to by the second mean.

[0110] In the above embodiment, since the corresponding image metric characterization vector can be obtained by only performing mean calculation and difference calculation on the second processing result, the efficiency of determining the image metric characterization vector can be greatly improved.

[0111] In one embodiment, a plurality of second difference results are integrated to obtain an image metric representation vector corresponding to a current training sample, including: performing a mean operation on the plurality of second difference results to obtain a second mean difference; performing a variance operation on the plurality of second difference results to obtain a second variance difference; determining a maximum value among the plurality of second difference results to obtain a second maximum difference; and integrating the second mean difference, the second variance difference and the second maximum difference to obtain an image metric representation vector for the current training sample.

[0112] Specifically, when multiple second difference results corresponding to the current training sample are obtained, the computer device can perform an average operation on the multiple second difference results to obtain a second mean difference. Furthermore, the computer device can perform a variance operation on the multiple second difference results to obtain a second variance difference. For example, the computer device can obtain the second variance difference by determining the difference between the second difference result and the second mean difference. And the computer device determines the maximum value among the multiple second difference results to obtain the second maximum difference, and combines the second mean difference, the second variance difference and the second maximum difference to obtain the image metric representation vector of the current training sample. It is easy to understand that for each training sample in the training sample set, the computer device can determine a set of second difference results UModel corresponding to each training sample. 1,…,T , where UModel i represents the second difference result obtained by the training sample in the i-th round of image recognition. Therefore, the computer device can determine the corresponding second mean difference, second variance difference and second maximum difference for each training sample, and then determine the image metric representation vector corresponding to each training sample based on the second mean difference, second variance difference and second maximum difference.

[0113] In one embodiment, the computer device may determine the image metric representation vector by the following formula: in, represents the model metric representation vector, mean(UModel) represents the second mean difference, std(UModel) represents the second variance difference, and max(UModel) represents the second maximum difference.

[0114] In the above embodiment, since the image metric characterization vector includes the metric average value, metric fluctuation degree and maximum fluctuation range information introduced due to image uncertainty, the subsequent comprehensive metric result determined based on multiple information can be more accurate and the information contained can be richer.

[0115] Step S210, synthesize the model metric representation vector and the image metric representation vector to obtain a comprehensive metric result, and filter out target training samples based on the comprehensive metric result of each training sample in the training sample set; the filtered target training samples are used to train the image processing model.

[0116] Specifically, for each training sample in the plurality of training samples, the computer device determines the model metric representation vector and the image metric representation vector of each training sample, and synthesizes the model metric representation vector and the image metric representation vector to obtain the comprehensive measurement result corresponding to each training sample. The comprehensive measurement result of the training sample reflects the potential uncertainty of the output result after the image processing model infers the training sample, and the difficulty of the training sample. The difficulty of the training sample refers to the difficulty of training the model through the training sample and achieving the preset training effect, that is, the difficult training sample refers to the difficult training sample, and the easy training sample refers to the easy training sample. The modulus length of the comprehensive measurement result is positively correlated with the uncertainty, wherein the uncertainty of a training sample includes model uncertainty and image uncertainty. It is easy to understand that the longer the modulus length of the comprehensive measurement result of the training sample, the greater the uncertainty of the training sample. When screening the training samples, if only the training samples with large uncertainty are selected, it will cause the image processing model to be difficult to train and fail to achieve the best effect, so that the longer the modulus length of the comprehensive measurement result of the training sample, the greater the difficulty of the training sample. Therefore, when the modulus of the comprehensive measurement result is long, the uncertainty of the training sample is greater, and the training sample is a difficult sample.

[0117] Furthermore, if only training samples with longer modulus lengths of the comprehensive metric results are selected, it will make the image processing model difficult to train and fail to achieve the best effect. Therefore, the computer device can filter out target training samples from the training sample set according to the comprehensive metric results corresponding to each training sample and the actual training requirements of the model. For example, the computer device can evenly filter out target training samples with different modulus lengths of the comprehensive metric results from the training sample set, so as to train the image processing model based on the evenly filtered target training samples.

[0118] In one embodiment, the computer device may concatenate the model metric representation vector and the image metric representation vector of the current training sample to obtain a comprehensive metric result of the current training sample. The image metric representation vector is When , the combined comprehensive measurement result can be

[0119]

[0120] In one embodiment, reference Figure 5 , Figure 5 A schematic diagram of a framework based on iterative model training in one embodiment is shown. The training sample set may include labeled training samples and unlabeled training samples. The computer device may perform reasoning on each labeled training sample and unlabeled training sample according to the above training sample screening method to obtain a comprehensive measurement result corresponding to each training sample. Furthermore, the computer device may screen out target training samples from unlabeled training samples according to the comprehensive measurement result, and perform label annotation processing on the target training samples to obtain labeled target training samples. The image processing model is trained by the labeled target training samples and the original labeled training samples. The above process is completed once as a cycle. In order to improve the accuracy of the results output by the image processing model, the above cycle iteration is usually performed regularly, that is, it is necessary to regularly screen out target training samples from unlabeled training samples and train the image processing model by the target training samples. In addition, since the training samples are processed by the image processing model itself to obtain the comprehensive measurement results of the training samples, the comprehensive measurement results corresponding to the same training sample are different for different models. Therefore, different target training samples will be screened out from the training sample set for different image processing models. In this way, personalized screening is achieved, and the training effect of the model is further improved based on the personalized screened target training samples.

[0121] In the above-mentioned training sample screening method, by randomly blocking the neurons in the image processing model, a blocking model can be obtained, so that the current training sample in the sample training set is image processed by the blocking model to obtain a first processing result, and a model metric characterization vector corresponding to the current training sample for characterizing model uncertainty is obtained based on the first processing result. By performing image transformation on the current training sample in the sample training set, a transformed sample can be obtained, so that the transformed sample is image processed by the image processing model to obtain a second processing result, and an image metric characterization vector corresponding to the current training sample for characterizing image uncertainty is obtained based on the second processing result. By obtaining the model metric characterization vector and the image metric characterization vector corresponding to each training sample in the sample training set, a comprehensive metric result for characterizing the difficulty of the training sample can be obtained based on the model metric characterization vector and the image metric characterization vector, so that the target training sample that is more conducive to model training can be screened out from the training sample set based on the comprehensive metric result. Since the present application can screen out the target training sample by the comprehensive metric result for characterizing the difficulty of the sample, compared with the traditional random screening of training samples, the present application can screen out valuable training samples, thereby improving the training effect of the image processing model based on valuable training samples.

[0122] In one embodiment, the training sample set includes a first training sample set and a second training sample set, the first training sample set includes a first training sample with a label; the second training sample set includes a second training sample without a label; a comprehensive measurement result is obtained by integrating a model metric representation vector and an image metric representation vector, including: respectively splicing the model metric representation vector and the image metric representation vector corresponding to each first training sample to obtain a first comprehensive measurement result corresponding to each first training sample; respectively splicing the model metric representation vector and the image metric representation vector corresponding to each second training sample to obtain a second comprehensive measurement result corresponding to each second training sample.

[0123] Specifically, the training sample set includes a first training sample set and a second training sample set, wherein the first training sample set stores a first training sample, which is a training sample with a label; the second training sample set stores a second training sample, which is a training sample without a label. When the model metric representation vector and the image metric representation vector corresponding to each first training sample are obtained, for each first training sample in the plurality of first training samples, the computer device splices the model metric representation vector and the image metric representation vector of the current first training sample to obtain the first comprehensive measurement result corresponding to the current first training sample. For example, the computer device may splice the image metric representation vector to the model metric representation vector to obtain the first comprehensive measurement result.

[0124] Correspondingly, when the model metric representation vector and the image metric representation vector corresponding to each second training sample are obtained, for each second training sample in the multiple second training samples, the computer device splices the model metric representation vector and the image metric representation vector of the current second training sample to obtain the second comprehensive measurement result corresponding to the current second training sample.

[0125] In the above embodiment, by splicing the model metric representation vector and the image metric representation vector, a comprehensive measurement result including model uncertainty and image uncertainty can be obtained, that is, a comprehensive measurement result for measuring uncertainty is obtained, so that target training samples that are beneficial to model training can be screened out from the training sample set based on the comprehensive measurement result for measuring uncertainty.

[0126] In one embodiment, reference Figure 6 , selecting target training samples according to the comprehensive measurement results of each training sample in the training sample set, including:

[0127] Step S602, for the first training sample set and the second training sample set corresponding to the current screening, respectively determine the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set, and obtain multiple measurement result differences corresponding to each second comprehensive measurement result.

[0128] Step S604: Filter out the target training sample for this time from the second training sample set according to the differences of the multiple measurement results corresponding to each second comprehensive measurement result.

[0129] Specifically, since the modulus of the comprehensive measurement result is positively correlated with the uncertainty, when screening the target training samples, if only training samples with large uncertainty are selected, the model will be difficult to train and cannot achieve the best effect. Therefore, in order to reduce the probability of selecting all target training samples with large uncertainty, so that the screened target training samples and the original first training samples are evenly distributed in the uncertainty space, the computer device can combine the distribution of the existing labeled first training samples in the uncertainty space to screen out the target training samples, so that the training effect of the model can be improved later through the evenly distributed target training samples and first training samples.

[0130] For the first training sample set and the second training sample set corresponding to the current screening, when multiple first comprehensive measurement results and multiple second comprehensive measurement results are obtained, the computer device can determine the difference between each first comprehensive measurement result and each second comprehensive measurement result to obtain multiple measurement result differences. For example, for each second comprehensive measurement result in multiple second comprehensive measurement results, the computer device determines the difference between the current second comprehensive measurement result and each first comprehensive measurement result to obtain multiple measurement result differences corresponding to the current second comprehensive measurement result. Among them, the measurement result difference refers to the difference between the second comprehensive measurement result and the first comprehensive measurement result. For example, when the second comprehensive measurement result of the second training sample A is a, the first comprehensive measurement result of the first training sample B is b, and the first comprehensive measurement result of the first training sample C is c, at this time, the multiple measurement result differences corresponding to the second comprehensive measurement result a include the difference between a and b and the difference between a and c.

[0131] Furthermore, the computer device selects the target training sample from the second training sample set according to the differences in the multiple measurement results, and obtains the target training sample selected this time. For example, the computer device may use the second training sample with the largest measurement result difference as the target training sample, or the computer device may use the second training sample with the smallest measurement result difference as the target training sample, or the computer device may use the second training sample with the target value measurement result as the target training sample, etc.

[0132] In one embodiment, the computer device may determine the minimum Euclidean distance or the minimum Manhattan distance between the second comprehensive measurement result and the first comprehensive measurement result, and use the minimum Euclidean distance between the second comprehensive measurement result and the first comprehensive measurement result as the measurement result difference, or use the minimum Manhattan distance between the second comprehensive measurement result and the first comprehensive measurement result as the measurement result difference.

[0133] In one of the embodiments, when multiple metric difference results are obtained, the multiple metric difference results can be clustered to obtain at least one cluster, so as to filter out the target metric difference from the cluster as required, and use the second training sample with the target metric difference as the target training sample.

[0134] Step S606, label the target training sample of the time, store the labeled target training sample into the first training sample set to obtain an updated first training sample set, and delete the target training sample in the second training sample set to obtain an updated second training sample set.

[0135] Step S608, using the updated first training sample set as the first training sample set corresponding to the next screening, using the updated second training sample set as the second training sample set corresponding to the next screening, entering the next screening and returning to respectively determine the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set, and obtaining the difference between multiple measurement results corresponding to each second comprehensive measurement result, and continue to execute until the third stop condition is reached to obtain the second preset number of target training samples.

[0136] Specifically, the computer device performs labeling processing on the target training samples screened this time, and stores the labeled target training samples in the first training set to obtain an updated first training sample set, and deletes the target training samples in the second training set to obtain an updated second training sample set. The computer device returns to the step of determining the difference between the first comprehensive metric result of each first training sample in the first training sample set and the second comprehensive metric result of each second training sample in the second training sample set, and continues to execute, and iterates until the third stop condition is reached, and a second preset number of target training samples is obtained.

[0137] In this embodiment, since the second training samples are automatically screened in combination with the comprehensive measurement results of the existing first training samples, the first training samples and the screened target training samples can be more evenly distributed in the uncertainty space, thereby improving the processing effect of the image processing model trained based on the evenly distributed first training samples and target training samples.

[0138] In addition, since the target training samples that are beneficial to model training are screened out from the second training sample set, compared with the traditional random screening of training samples, the embodiment of the present application can reduce the probability of labeling the screened redundant training samples, thereby improving the labeling efficiency.

[0139] In one embodiment, before respectively determining the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set, and obtaining multiple measurement result differences corresponding to each second comprehensive measurement result, the method also includes: normalizing the first comprehensive measurement result to obtain a first normalized result, and mapping the first normalized result through a mapping function to obtain a first mapping result; normalizing the second comprehensive measurement result to obtain a second normalized result, and mapping the second normalized result through a mapping function to obtain a second mapping result; respectively determining the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set, and obtaining multiple measurement result differences corresponding to each second comprehensive measurement result, including: respectively determining the difference between the first mapping result corresponding to each first training sample in the first training sample set and the second mapping result corresponding to each second training sample in the second training sample set, and obtaining multiple measurement result differences corresponding to each second comprehensive measurement result.

[0140] Specifically, in order to reduce the probability that the algorithm selects all of the difficult second training samples due to their excessive dispersion in the uncertainty space, it is necessary to normalize and saturate map the first comprehensive metric result and the second comprehensive metric result.

[0141] When the first comprehensive measurement result is obtained, the computer device may normalize the first comprehensive measurement result to obtain a first normalized result, and map the first normalized result through a preset mapping function to obtain a first mapping result. Correspondingly, the computer device may normalize the second comprehensive measurement result to obtain a second normalized result, and map the second normalized result through a preset mapping function to obtain a second mapping result. Further, for each second mapping result in the plurality of second mapping results, the computer device determines the difference between the current second mapping result and each first mapping result, and uses the difference as a measurement result difference, thereby obtaining a plurality of measurement result differences corresponding to the current second mapping result.

[0142] In one embodiment, the computer device may normalize the first comprehensive measurement result by the following formula to obtain a first normalized result:

[0143]

[0144] in, is the first comprehensive measurement result, is the mean of multiple first comprehensive measurement results, is the variance of the plurality of first comprehensive measurement results. Similarly, the computer device may also perform normalization processing on the second comprehensive measurement result by using the above formula.

[0145] In one embodiment, the computer device may map the first normalization result by the following formula to obtain a first mapping result:

[0146]

[0147]

[0148] Among them, T is the Sigmoid slope control coefficient. By increasing T, more difficult training samples will be compressed into the saturation area, thereby reducing the probability of their selection. Therefore, T can be used to control the ratio between training samples with a difficulty greater than a preset value and training samples with a difficulty less than a preset value, that is, to control the screening ratio of difficult and easy samples, thereby further improving the training effect of the model by controlling the screening ratio of difficult and easy samples. It is easy to understand that the computer device can also determine the second normalized result through the above formula. Both normalization and function mapping are bit operations, that is, each element in the first comprehensive measurement result and the second comprehensive measurement result is independently operated.

[0149] In the above embodiment, by normalizing and saturating the first comprehensive measurement result and the second comprehensive measurement result, the dispersion of training samples with a difficulty greater than a preset value in the uncertainty space can be reduced, so that training samples with a difficulty greater than the preset value and training samples with a difficulty less than or equal to the preset value can be evenly selected.

[0150] In one embodiment, based on the multiple measurement result differences corresponding to each second comprehensive measurement result, the target training sample of the time is screened out from the second training sample set, including: taking the minimum measurement result difference among the multiple measurement result differences corresponding to each second comprehensive measurement result as the target measurement result difference of the corresponding second comprehensive measurement result; screening out the maximum target measurement result difference from the multiple target measurement result differences, and taking the second comprehensive measurement result with the maximum target measurement result difference as the target second comprehensive measurement result; taking the second training sample with the target second comprehensive measurement result as the target training sample.

[0151] Specifically, the computer device may determine the target measurement result difference corresponding to each second comprehensive measurement result, wherein, for each second comprehensive measurement result among the multiple second comprehensive measurement results, the computer device uses the minimum measurement result difference among the multiple measurement result differences corresponding to the current second comprehensive measurement result as the target measurement result difference corresponding to the current second comprehensive measurement result. Further, the computer device screens out the maximum target measurement result difference from the target measurement result differences corresponding to each second comprehensive measurement result, and uses the second comprehensive measurement result with the maximum target measurement result difference as the target second comprehensive measurement result, and uses the second training sample with the target second comprehensive measurement result as the target training sample. For example, when the second comprehensive measurement result a of the second training sample A is obtained, and the target measurement result difference corresponding to the second comprehensive measurement result a is the maximum target measurement result difference, the second training sample A is used as the target training sample.

[0152] In one embodiment, the computer device may determine the target measurement result difference by the following formula: d=min j∈m Δ(u,s j ), where u is the second comprehensive measurement result, s j is the first comprehensive measurement result, m is the first comprehensive measurement result set, and the first comprehensive measurement result set includes the first comprehensive measurement result corresponding to each first training sample. Δ is the Manhattan distance or the Euclidean distance.

[0153] In the above embodiment, since the target training samples are screened out based on the difference in measurement results, the screened target training samples can be more uniform in the uncertain space, thereby improving the training effect and training efficiency of the model based on the uniformly screened target training samples.

[0154] In one embodiment, the training sample set includes a first training sample set, and the first training sample set includes a first training sample with a labeled label. The above method also includes: labeling the target training sample, storing the labeled target training sample in the first training sample set, and obtaining an updated first training sample set; training the image processing model with the updated first training sample set until a fourth preset stop condition is reached, and obtaining a trained image processing model; the trained image processing model is used to identify the image to be identified.

[0155] Specifically, when the target training sample is obtained, the computer device may perform labeling processing on the target training sample to obtain the target training sample with the label, and store the target training sample with the label in the first sample set. When the target training sample is stored in the first training sample set, the computer device may train the image processing model through the first training sample until the fourth preset stop condition is reached, and obtain the trained image processing model. The trained image processing model can be used to process the image, for example, to identify the target object in the image, or to segment the image, or to determine the key points in the image, etc.

[0156] In this embodiment, by further training the image processing model, the trained image processing model can output more accurate processing results.

[0157] The present application also provides an application scenario, which applies the above-mentioned training sample screening method.

[0158] Specifically, the application of the training sample screening method in this application scenario is as follows:

[0159] In the field of smart retail, the image processing model can be used to identify videos entering and exiting the mall, obtain the customer's entry and exit trajectory, and obtain the mall's customer flow through the entry and exit trajectory. Since the mall environment changes at regular intervals, the image processing model needs to be optimized and trained regularly. R&D personnel can obtain multiple labeled first training samples and multiple unlabeled second training samples, so that the computer device can filter out a preset number of target training samples from multiple second training samples according to the above training sample screening method, and optimize the image processing model through the first training samples and the screened target training samples to obtain an optimized and trained image processing model.

[0160] The present application also provides an application scenario, which applies the above-mentioned training sample screening method. Specifically, the application of the training sample screening method in the application scenario is as follows:

[0161] In face recognition scenarios, in order to improve the security of resource transfer, before transferring resources through applications, liveness detection can be performed on face images including user faces through image processing models. As science and technology develop, face forgery technology will also improve accordingly. Therefore, it is necessary to regularly optimize and train image processing models to improve the accuracy of liveness detection based on the optimized and trained image processing models. The computer device can select a preset number of target training samples from the sample training set according to the above training sample screening method, and optimize and train the image processing model through the selected target training samples to obtain an optimized and trained image processing model.

[0162] The above application scenarios are merely illustrative descriptions. It is to be understood that the application of the training sample screening methods provided in the embodiments of the present application is not limited to the above scenarios.

[0163] It should be noted that the user information (including but not limited to user device information, user personal information, user entry and exit trajectories, user facial images, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0164] In a specific embodiment, referring to Figure 7 , provides a training sample screening method, the training sample screening method comprising:

[0165] S702, for each training sample in a plurality of training samples, the computer device randomly blocks a first preset number of neurons in the image recognition model to obtain a blocking model, and performs image recognition on the current training sample through the blocking model to obtain a first processing result, enters the next round of image recognition, and returns to the step of randomly blocking the first preset number of neurons in the image recognition model to continue executing until a first preset stop condition is met, thereby obtaining a plurality of first processing results corresponding to the current training sample.

[0166] S704: For each training sample in the multiple training samples, the computer device performs an average operation on multiple first processing results corresponding to the current training sample to obtain a first average result corresponding to the current training sample.

[0167] S706, the computer device determines the difference between each first processing result corresponding to the current training sample and the first mean result, obtains the first difference result corresponding to each first processing result, performs a mean operation on the multiple first difference results to obtain a first mean difference, performs a variance operation on the multiple first difference results to obtain a first variance difference, determines the maximum value among the multiple first difference results, and obtains a first maximum difference.

[0168] S708, the computer device integrates the first mean difference, the first variance difference and the first maximum difference to obtain a model metric representation vector of the current training sample.

[0169] S710, the computer device adjusts at least one of the brightness, contrast, saturation and sharpness of each training sample respectively to obtain a color perturbation sample corresponding to each training sample; performs pixel displacement on the pixel points in each color perturbation sample respectively to obtain a transformed sample corresponding to each training sample, enters the next round of image transformation, returns to the step of adjusting at least one of the brightness, contrast, saturation and sharpness of each training sample respectively, and continues to execute until a second preset stop condition is met, and a plurality of transformed samples corresponding to each training sample in the training sample set are obtained.

[0170] S712: For each training sample in the multiple training samples, the computer device performs image processing on each transformed sample corresponding to the current training sample through the image processing model to obtain multiple second processing results corresponding to the current training sample.

[0171] S714: For each training sample in the multiple training samples, the computer device performs an average operation on multiple second processing results corresponding to the current training sample to obtain a second average result corresponding to the current training sample.

[0172] S716, the computer device determines the difference between each second processing result corresponding to the current training sample and the second mean result, and obtains the second difference result corresponding to each second processing result; performs a mean operation on the multiple second difference results to obtain a second mean difference, performs a variance operation on the multiple second difference results to obtain a second variance difference, determines the maximum value among the multiple second difference results, and obtains the second maximum difference.

[0173] S718, the computer device combines the second mean difference, the second variance difference, and the second maximum difference to obtain an image metric representation vector of the current training sample.

[0174] S720, the computer device concatenates the model metric representation vector and the image metric representation vector corresponding to each first training sample, to obtain a first comprehensive metric result corresponding to each first training sample.

[0175] S722, the computer device concatenates the model metric representation vector and the image metric representation vector corresponding to each second training sample, respectively, to obtain a second comprehensive metric result corresponding to each second training sample.

[0176] S724, the computer device respectively determines the difference between each first comprehensive measurement result and each second comprehensive measurement result, obtains multiple measurement result differences corresponding to each second comprehensive measurement result, and takes the minimum measurement result difference among the multiple measurement result differences corresponding to each second comprehensive measurement result as the target measurement result difference of the corresponding second comprehensive measurement result.

[0177] S726, the computer device selects the maximum target measurement result difference from the multiple target measurement result differences, and uses the second comprehensive measurement result with the maximum target measurement result difference as the target second comprehensive measurement result, and uses the second training sample with the target second comprehensive measurement result as the target training sample.

[0178] S728, the computer device labels the target training samples, stores the labeled target training samples in the first training sample set, deletes the target training samples in the second training sample set, returns to the step of splicing the model metric representation vector and the image metric representation vector corresponding to each first training sample, and continues to execute until the third stop condition is reached, thereby obtaining a second preset number of target training samples.

[0179] The above-mentioned training sample screening method can obtain a blocking model by randomly blocking neurons in the image processing model, thereby performing image processing on the training sample through the blocking model to obtain a first processing result, and obtaining a model metric representation vector for characterizing model uncertainty based on the first processing result. By performing image transformation on the training sample, a transformed sample can be obtained, thereby performing image processing on the transformed sample through the image processing model to obtain a second processing result, and obtaining an image metric representation vector for characterizing image uncertainty based on the second processing result. By obtaining the model metric representation vector and the image metric representation vector, a comprehensive measurement result for characterizing the difficulty of the training sample can be obtained based on the model metric representation vector and the image metric representation vector. In this way, target training samples that are more conducive to model training can be screened out from the training sample set based on the comprehensive measurement result. Since the present application can screen out target training samples through a comprehensive measurement result for characterizing the difficulty of the sample, compared with the traditional random screening of training samples, the present application can screen out valuable training samples, thereby improving the training effect of the image processing model based on valuable training samples.

[0180] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0181] In one embodiment, Figure 8 As shown, a training sample screening device 800 is provided. The device can adopt a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes: a model uncertainty acquisition module 802, an image uncertainty acquisition module 804 and a screening module 806, wherein:

[0182] The model uncertainty acquisition module 802 is used to randomly block neurons in the image processing model to obtain a blocking model, and perform image processing on the current training sample in the training sample set through the blocking model to obtain a first processing result; for the current training sample, based on the first processing results corresponding to multiple random blockages, the corresponding model metric representation vector is determined, and the model metric representation vector is used to characterize the model uncertainty.

[0183] The image uncertainty acquisition module 804 is used to obtain a transformed sample obtained by performing image transformation on the current training sample, and perform image processing on the transformed sample through an image processing model to obtain a second processing result; for the current training sample, based on the second processing results of the corresponding multiple transformed samples, a corresponding image metric representation vector is determined, and the image metric representation vector is used to characterize the image uncertainty.

[0184] The screening module 806 is used to obtain a comprehensive measurement result by integrating the model measurement representation vector and the image measurement representation vector, and screen out target training samples according to the comprehensive measurement result of each training sample in the training sample set; the screened target training samples are used to train the image processing model.

[0185] The above-mentioned training sample screening device can obtain a blocking model by randomly blocking neurons in the image processing model, thereby performing image processing on the training sample through the blocking model to obtain a first processing result, and obtain a model metric characterization vector for characterizing model uncertainty based on the first processing result. By performing image transformation on the training sample, a transformed sample can be obtained, thereby performing image processing on the transformed sample through the image processing model to obtain a second processing result, and obtain an image metric characterization vector for characterizing image uncertainty based on the second processing result. By obtaining the model metric characterization vector and the image metric characterization vector, a comprehensive metric result for characterizing the difficulty of the training sample can be obtained based on the model metric characterization vector and the image metric characterization vector, so that the target training sample that is more conducive to model training can be screened out from the training sample set based on the comprehensive metric result. Since the present application can screen out target training samples through a comprehensive metric result for characterizing the difficulty of the sample, compared with the traditional random screening of training samples, the present application can screen out valuable training samples, thereby improving the training effect of the image processing model based on valuable training samples.

[0186] In one embodiment, reference Fig. 9 The model uncertainty acquisition module 802 also includes a first processing result acquisition module 8021, which is used to randomly block a first preset number of neurons in the image recognition model for the current training sample in the training sample set to obtain a blocking model, and perform image recognition on the current training sample through the blocking model to obtain a first processing result; enter the next round of image recognition, return to the step of randomly blocking the first preset number of neurons in the image recognition model and continue to execute until the first preset stop condition is met, and obtain multiple first processing results corresponding to the current training sample.

[0187] In one embodiment, the model uncertainty acquisition module 802 is also used to perform an averaging operation on multiple first processing results corresponding to multiple random blockages for the current training sample to obtain a first mean result corresponding to the current training sample; determine the difference between each first processing result corresponding to the current training sample and the first mean result to obtain a first difference result corresponding to each first processing result; and combine multiple first difference results to obtain a model metric representation vector of the current training sample.

[0188] In one embodiment, the model uncertainty acquisition module 802 is used to perform a mean operation on multiple first difference results to obtain a first mean difference; perform a variance operation on multiple first difference results to obtain a first variance difference; determine the maximum value among the multiple first difference results to obtain a first maximum difference; and combine the first mean difference, the first variance difference and the first maximum difference to obtain a model metric representation vector of the current training sample.

[0189] In one embodiment, the image uncertainty acquisition module 804 also includes an image transformation module 8041, which is used to adjust at least one of the brightness, contrast, saturation and sharpness of the current training sample to obtain a color perturbation sample corresponding to the current training sample; perform pixel displacement on the pixel points in the color perturbation sample to obtain a transformed sample corresponding to the current training sample; enter the next round of image transformation, return to the step of adjusting at least one of the brightness, contrast, saturation and sharpness of the current training sample and continue to execute until the second preset stop condition is met, and obtain multiple transformed samples corresponding to the current training sample.

[0190] In one embodiment, the image uncertainty acquisition module 804 is also used to perform an averaging operation on multiple second processing results corresponding to the current training sample to obtain a second mean result corresponding to the current training sample; determine the difference between each second processing result corresponding to the current training sample and the second mean result to obtain a second difference result corresponding to each second processing result; and combine the multiple second difference results to obtain an image metric representation vector corresponding to the current training sample.

[0191] In one embodiment, the image uncertainty acquisition module 804 is also used to perform a mean operation on multiple second difference results to obtain a second mean difference; perform a variance operation on multiple second difference results to obtain a second variance difference; determine the maximum value among the multiple second difference results to obtain a second maximum difference; and combine the second mean difference, the second variance difference and the second maximum difference to obtain an image metric representation vector of the current training sample.

[0192] In one embodiment, the training sample set includes a first training sample set and a second training sample set, the first training sample set includes a first training sample with a label; the second training sample set includes a second training sample without a label; the screening module 806 is also used to respectively splice the model metric representation vector and the image metric representation vector corresponding to each first training sample to obtain a first comprehensive measurement result corresponding to each first training sample; and respectively splice the model metric representation vector and the image metric representation vector corresponding to each second training sample to obtain a second comprehensive measurement result corresponding to each second training sample.

[0193] In one embodiment, the screening module 806 is further used to determine the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set for the first training sample set and the second training sample set corresponding to the current screening, and obtain a plurality of measurement result differences corresponding to each second comprehensive measurement result; screen out the current target training sample from the second training sample set according to the plurality of measurement result differences corresponding to each second comprehensive measurement result; label the current target training sample, store the labeled target training sample in the first training sample set, and obtain an updated first training sample set. The method further comprises the following steps: obtaining a first training sample set and deleting the target training sample in the second training sample set to obtain an updated second training sample set; using the updated first training sample set as the first training sample set corresponding to the next screening, and using the updated second training sample set as the second training sample set corresponding to the next screening, entering the next screening and returning to respectively determine the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set, and obtaining the difference between multiple measurement results corresponding to each second comprehensive measurement result, and continuing to execute the step until the third stopping condition is reached to obtain a second preset number of target training samples.

[0194] In one embodiment, the screening module 806 also includes a mapping module 8061, which is used to normalize the first comprehensive measurement result to obtain a first normalized result, and map the first normalized result through a mapping function to obtain a first mapping result; normalize the second comprehensive measurement result to obtain a second normalized result, and map the second normalized result through a mapping function to obtain a second mapping result; respectively determine the difference between the first mapping result corresponding to each first training sample in the first training sample set and the second mapping result corresponding to each second training sample in the second training sample set, and obtain multiple measurement result differences corresponding to each second comprehensive measurement result.

[0195] In one embodiment, the screening module 806 also includes a target determination module 8062, which is used to use the minimum measurement result difference among the multiple measurement result differences corresponding to each second comprehensive measurement result as the target measurement result difference of the corresponding second comprehensive measurement result; screen out the maximum target measurement result difference from the multiple target measurement result differences, and use the second comprehensive measurement result with the maximum target measurement result difference as the target second comprehensive measurement result; and use the second training sample with the target second comprehensive measurement result as the target training sample.

[0196] In one embodiment, the training sample set includes a first training sample set, which includes a first training sample with a labeled label. The training sample screening device 800 is also used to label the target training samples, and store the labeled target training samples in the first training sample set to obtain an updated first training sample set; the image processing model is trained with the updated first training sample set until a fourth preset stop condition is reached to obtain a trained image processing model; the trained image processing model is used to identify the image to be identified.

[0197] For the specific definition of the training sample screening device, please refer to the definition of the training sample screening method above, which will not be repeated here. Each module in the above-mentioned training sample screening device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0198] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store training samples. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a training sample screening method is implemented.

[0199] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0200] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0201] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0202] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0203] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0204] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0205] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A training sample screening method, characterized in that: The method comprises: Randomly blocking neurons in the image processing model to obtain a blocking model, and performing image processing on a current training sample in a training sample set by using the blocking model to obtain a first processing result; For the current training sample, based on first processing results corresponding to the multiple random blockings respectively, determining a corresponding model metric characterization vector, wherein the model metric characterization vector is used to characterize model uncertainty; Acquire a transformed sample obtained by performing image transformation on the current training sample, and perform image processing on the transformed sample through the image processing model to obtain a second processing result; For the current training sample, based on the second processing results of the corresponding plurality of transformed samples, determining a corresponding image metric representation vector, wherein the image metric representation vector is used to represent image uncertainty; The model metric representation vector and the image metric representation vector are combined to obtain a comprehensive metric result, and a target training sample is screened out according to the comprehensive metric result of each training sample in the training sample set; the screened target training sample is used to train the image processing model; The step of determining, for the current training sample, a corresponding model metric characterization vector based on first processing results corresponding to the multiple random blockings respectively, includes: For the current training sample, performing an average operation on a plurality of first processing results corresponding to a plurality of random blockings to obtain a first average result corresponding to the current training sample; Determine the difference between each first processing result corresponding to the current training sample and the first mean result, and obtain a first difference result corresponding to each first processing result; Combining the plurality of first difference results to obtain a model metric representation vector of the current training sample; The step of determining, for the current training sample, a corresponding image metric representation vector based on the second processing results of the corresponding plurality of transformed samples comprises: For the current training sample, performing an average operation on a plurality of second processing results corresponding to the current training sample to obtain a second average result corresponding to the current training sample; Determine the difference between each second processing result corresponding to the current training sample and the second mean result, and obtain a second difference result corresponding to each second processing result; A plurality of the second difference results are combined to obtain an image metric representation vector corresponding to the current training sample.

2. The method according to claim 1, characterized in that The neurons in the randomly blocked image processing model are obtained by obtaining a blocking model, and the current training sample in the training sample set is image processed by using the blocking model to obtain a first processing result, including: For a current training sample in the training sample set, randomly blocking a first preset number of neurons in the image processing model to obtain a blocking model, and performing image recognition on the current training sample by using the blocking model to obtain a first processing result; Entering the next round of image recognition, returning to the step of randomly blocking a first preset number of neurons in the image processing model and continuing to execute until a first preset stop condition is met, and obtaining multiple first processing results corresponding to the current training sample.

3. The method according to claim 1, characterized in that: The step of synthesizing the plurality of first difference results to obtain a model metric representation vector of the current training sample includes: Performing an average operation on the plurality of first difference results to obtain a first mean difference; Performing a variance operation on the plurality of first difference results to obtain a first variance difference; Determine a maximum value among a plurality of the first difference results to obtain a first maximum difference; The first mean difference, the first variance difference and the first maximum difference are combined to obtain a model metric representation vector of the current training sample.

4. The method according to claim 1, characterized in that: The method further comprises: Adjust at least one of the brightness, contrast, saturation and sharpness of the current training sample to obtain a color perturbation sample corresponding to the current training sample; Performing pixel shift on the pixel points in the color disturbance sample to obtain a transformed sample corresponding to the current training sample; Entering the next round of image transformation, returning to the step of adjusting at least one of the brightness, contrast, saturation and sharpness of the current training sample, and continuing to execute until a second preset stop condition is met, and obtaining multiple transformed samples corresponding to the current training sample.

5. The method according to claim 1, characterized in that The step of synthesizing the plurality of second difference results to obtain an image metric representation vector corresponding to the current training sample includes: Performing an average operation on the plurality of second difference results to obtain a second mean difference; performing a variance operation on the plurality of second difference results to obtain a second variance difference; Determine a maximum value among a plurality of the second difference results to obtain a second maximum difference; The image metric representation vector of the current training sample is obtained by combining the second mean difference, the second variance difference and the second maximum difference.

6. The method according to claim 1, characterized in that The training sample set includes a first training sample set and a second training sample set, wherein the first training sample set includes a first training sample with a label; The second training sample set includes a second training sample without a label; The step of synthesizing the model metric representation vector and the image metric representation vector to obtain a comprehensive metric result includes: Respectively concatenating the model metric representation vector and the image metric representation vector corresponding to each of the first training samples to obtain a first comprehensive metric result corresponding to each of the first training samples; The model metric representation vector and the image metric representation vector corresponding to each of the second training samples are concatenated to obtain a second comprehensive metric result corresponding to each of the second training samples.

7. The method according to claim 6, characterized in that The step of selecting a target training sample according to the comprehensive measurement result of each training sample in the training sample set includes: For the first training sample set and the second training sample set corresponding to the current screening, respectively determine the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set, and obtain a plurality of measurement result differences corresponding to each second comprehensive measurement result; According to the differences of the multiple measurement results corresponding to each of the second comprehensive measurement results, the target training sample for the current time is selected from the second training sample set; Labeling the target training sample of the current time, storing the labeled target training sample in the first training sample set to obtain an updated first training sample set, and deleting the target training sample in the second training sample set to obtain an updated second training sample set; The updated first training sample set is used as the first training sample set corresponding to the next screening, and the updated second training sample set is used as the second training sample set corresponding to the next screening, and the next screening is entered and the step of respectively determining the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set, and obtaining the difference of multiple measurement results corresponding to each second comprehensive measurement result is continued until the third stopping condition is reached, and the second preset number of target training samples is obtained.

8. The method according to claim 7, characterized in that Before respectively determining the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set, and obtaining a plurality of measurement result differences corresponding to each second comprehensive measurement result, the method further includes: Normalizing the first comprehensive measurement result to obtain a first normalized result, and mapping the first normalized result through a mapping function to obtain a first mapping result; Normalizing the second comprehensive measurement result to obtain a second normalized result, and mapping the second normalized result through a mapping function to obtain a second mapping result; The step of respectively determining the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set to obtain a plurality of measurement result differences corresponding to each second comprehensive measurement result includes: The differences between the first mapping result corresponding to each first training sample in the first training sample set and the second mapping result corresponding to each second training sample in the second training sample set are determined respectively, and a plurality of measurement result differences corresponding to each second comprehensive measurement result are obtained.

9. The method according to claim 7, characterized in that: The step of selecting a target training sample for the current time from the second training sample set according to the differences of the multiple measurement results corresponding to each of the second comprehensive measurement results includes: Taking the minimum measurement result difference among the multiple measurement result differences corresponding to each of the second comprehensive measurement results as the target measurement result difference of the corresponding second comprehensive measurement result; Screening out the maximum target measurement result difference from the multiple target measurement result differences, and using the second comprehensive measurement result with the maximum target measurement result difference as the target second comprehensive measurement result; The second training sample having the target second comprehensive metric result is used as the target training sample.

10. The method according to any one of claims 1 to 9, characterized in that The training sample set includes a first training sample set, wherein the first training sample set includes a first training sample with annotated labels, and the method further includes: Labeling the target training samples, storing the labeled target training samples into the first training sample set, and obtaining an updated first training sample set; The image processing model is trained using the updated first training sample set until a fourth preset stop condition is reached, thereby obtaining a trained image processing model; the trained image processing model is used to identify the image to be identified.

11. A training sample screening device, characterized in that: The device comprises: A model uncertainty acquisition module is used to randomly block neurons in an image processing model to obtain a blocking model, and perform image processing on a current training sample in a training sample set through the blocking model to obtain a first processing result; for the current training sample, perform an average operation on multiple first processing results corresponding to multiple random blockages to obtain a first mean result corresponding to the current training sample; determine the difference between each first processing result corresponding to the current training sample and the first mean result to obtain a first difference result corresponding to each first processing result; and obtain a model metric representation vector of the current training sample by combining multiple first difference results, wherein the model metric representation vector is used to represent model uncertainty; An image uncertainty acquisition module is used to obtain a transformed sample obtained by performing image transformation on the current training sample, and perform image processing on the transformed sample through the image processing model to obtain a second processing result; for the current training sample, perform an average operation on multiple second processing results corresponding to the current training sample to obtain a second mean result corresponding to the current training sample; determine the difference between each second processing result corresponding to the current training sample and the second mean result to obtain a second difference result corresponding to each second processing result; and synthesize multiple second difference results to obtain an image metric representation vector corresponding to the current training sample, wherein the image metric representation vector is used to represent image uncertainty; A screening module is used to synthesize the model metric representation vector and the image metric representation vector to obtain a comprehensive metric result, and screen out target training samples according to the comprehensive metric result of each training sample in the training sample set; the screened target training samples are used to train the image processing model.

12. The device according to claim 11, characterized in that The model uncertainty acquisition module also includes a first processing result acquisition module, which is used to randomly block a first preset number of neurons in the image processing model for a current training sample in the training sample set to obtain a blocking model, and perform image recognition on the current training sample through the blocking model to obtain a first processing result; enter the next round of image recognition, return to the step of randomly blocking the first preset number of neurons in the image processing model and continue to execute until a first preset stop condition is met, and obtain multiple first processing results corresponding to the current training sample.

13. The device according to claim 11, characterized in that The model uncertainty acquisition module is also used to perform a mean operation on multiple first difference results to obtain a first mean difference; perform a variance operation on multiple first difference results to obtain a first variance difference; determine the maximum value among multiple first difference results to obtain a first maximum difference; and combine the first mean difference, the first variance difference and the first maximum difference to obtain a model metric representation vector of the current training sample.

14. The device according to claim 11, characterized in that The image uncertainty acquisition module further includes an image transformation module, which is used to adjust at least one of the brightness, contrast, saturation and sharpness of the current training sample to obtain a color perturbation sample corresponding to the current training sample; Perform pixel shift on the pixel points in the color disturbance sample to obtain a transformed sample corresponding to the current training sample; enter the next round of image transformation, return to the step of adjusting at least one of the brightness, contrast, saturation and sharpness of the current training sample, and continue to execute until a second preset stop condition is met, and obtain multiple transformed samples corresponding to the current training sample.

15. The device according to claim 11, characterized in that The image uncertainty acquisition module is also used to perform an averaging operation on multiple second difference results to obtain a second mean difference; perform a variance operation on multiple second difference results to obtain a second variance difference; determine the maximum value among multiple second difference results to obtain a second maximum difference; and combine the second mean difference, the second variance difference and the second maximum difference to obtain the image metric representation vector of the current training sample.

16. The device according to claim 11, characterized in that The training sample set includes a first training sample set and a second training sample set, wherein the first training sample set includes first training samples with labels; and the second training sample set includes second training samples without labels; The screening module is further used to respectively concatenate the model metric representation vector and the image metric representation vector corresponding to each of the first training samples to obtain a first comprehensive metric result corresponding to each of the first training samples; The model metric representation vector and the image metric representation vector corresponding to each of the second training samples are concatenated to obtain a second comprehensive metric result corresponding to each of the second training samples.

17. The device according to claim 16, characterized in that The screening module is further used to determine, for the first training sample set and the second training sample set corresponding to the current screening, the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set, and obtain a plurality of measurement result differences corresponding to each second comprehensive measurement result; According to the differences of the multiple measurement results corresponding to each of the second comprehensive measurement results, the target training sample for the current time is selected from the second training sample set; Labeling the target training sample of the current time, storing the labeled target training sample in the first training sample set to obtain an updated first training sample set, and deleting the target training sample in the second training sample set to obtain an updated second training sample set; The updated first training sample set is used as the first training sample set corresponding to the next screening, and the updated second training sample set is used as the second training sample set corresponding to the next screening, and the next screening is entered and the step of respectively determining the difference between the first comprehensive measurement result of each first training sample in the first training sample set and the second comprehensive measurement result of each second training sample in the second training sample set, and obtaining the difference of multiple measurement results corresponding to each second comprehensive measurement result is continued until the third stopping condition is reached, and the second preset number of target training samples is obtained.

18. The device according to claim 17, characterized in that The screening module further includes a mapping module, which is used to perform normalization processing on the first comprehensive measurement result to obtain a first normalization result, and perform mapping processing on the first normalization result through a mapping function to obtain a first mapping result; Normalizing the second comprehensive measurement result to obtain a second normalized result, and mapping the second normalized result through a mapping function to obtain a second mapping result; The differences between the first mapping result corresponding to each first training sample in the first training sample set and the second mapping result corresponding to each second training sample in the second training sample set are determined respectively, and a plurality of measurement result differences corresponding to each second comprehensive measurement result are obtained.

19. The device according to claim 17, characterized in that The screening module further includes a target determination module, which is used to use the minimum measurement result difference among the multiple measurement result differences corresponding to each of the second comprehensive measurement results as the target measurement result difference of the corresponding second comprehensive measurement result; Screening out the maximum target measurement result difference from the multiple target measurement result differences, and using the second comprehensive measurement result with the maximum target measurement result difference as the target second comprehensive measurement result; The second training sample having the target second comprehensive metric result is used as the target training sample.

20. The device according to any one of claims 11 to 19, characterized in that The training sample set includes a first training sample set, the first training sample set includes a first training sample with a labeled label, the training sample screening device is further used to label the target training sample, store the labeled target training sample in the first training sample set, and obtain an updated first training sample set; The image processing model is trained using the updated first training sample set until a fourth preset stop condition is reached, thereby obtaining a trained image processing model; the trained image processing model is used to identify the image to be identified.

21. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

22. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

23. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images