Training data processing methods, devices and equipment

Abstract

This application discloses a training data processing method, apparatus, and device, applicable to the field of artificial intelligence. The method includes: acquiring at least two images with the same classification label identified by a first classification recognition model, as well as image feature information and confidence levels corresponding to the at least two images; determining association information between the at least two images based on the image feature information and confidence levels; wherein the association information represents the correlation between the at least two images; inputting the image feature information and association information corresponding to the at least two images into a data cleaning model for data cleaning to obtain a target image with an incorrect classification label among the at least two images; wherein the target image is used to train a second classification recognition model. The technical solution provided by the embodiments of this application can at least automate the training data processing.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training data processing method, apparatus, and device. Background Technology

[0002] Artificial intelligence software technology mainly includes several major areas such as computer vision, speech processing, natural language processing, and machine learning. Within machine learning, the application of artificial intelligence software technology includes data classification and recognition.

[0003] Currently, to improve the classification and recognition capabilities of classification and recognition models, one approach is to manually clean the data that the model misidentifies and then use this incorrect data to train the model. However, this manual data cleaning method consumes a significant amount of manpower and time, resulting in an excessive workload and low efficiency. Summary of the Invention

[0004] This application provides a training data processing method, apparatus, device, and computer-readable storage medium, which can at least solve the problems of the inability to automate data cleaning, the consumption of a large amount of manpower and time, and the low efficiency in related technologies.

[0005] On the one hand, this application provides a training data processing method, the method comprising:

[0006] Obtain at least two images with the same classification label identified by the first classification recognition model, as well as the image feature information and confidence level corresponding to the at least two images;

[0007] Based on the image feature information and confidence level corresponding to the at least two images, the association information of the at least two images is determined; wherein, the association information characterizes the correlation between the at least two images;

[0008] The image feature information and the association information corresponding to the at least two images are input into the data cleaning model for data cleaning to obtain the target image with incorrect classification label in the at least two images; wherein, the data cleaning model is trained based on the second training data with the same sample classification label identified by the first classification recognition model, and the sample feature information and sample confidence corresponding to the second training data; wherein, the target image is used to train the second classification recognition model.

[0009] On the other hand, this application also provides a training data processing apparatus, the apparatus comprising:

[0010] The acquisition module is used to acquire at least two images with the same classification label identified by the first classification recognition model, as well as the image feature information and confidence level corresponding to the at least two images;

[0011] The determining module is configured to determine the association information of the at least two images based on the image feature information and confidence level corresponding to the at least two images; wherein the association information characterizes the correlation between the at least two images;

[0012] The data cleaning module is used to input the image feature information and the association information corresponding to the at least two images into the data cleaning model for data cleaning to obtain the target image with incorrect classification label in the at least two images; wherein, the data cleaning model is trained based on the second training data with the same sample classification label identified by the first classification recognition model, and the sample feature information and sample confidence corresponding to the second training data; wherein, the target image is used to train the second classification recognition model.

[0013] On the other hand, this application also provides a training data processing device, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the training data processing method as described above.

[0014] On the other hand, this application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the training data processing method described above.

[0015] On the other hand, this application also provides a storage medium that can be disposed in a server to store at least one instruction, at least one program, code set or instruction set related to implementing a training data processing method in the method embodiment. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the training data processing method provided in the above method embodiment.

[0016] The training data processing method, apparatus, device, and computer-readable storage medium provided in this application have the following technical effects:

[0017] This application utilizes a data cleaning model to clean at least two images with the same classification label identified by a first classification recognition model, thereby obtaining a target image with an incorrect classification label from at least two images. The target image is then used to train a second classification recognition model, which can automate the training data processing, save a lot of manpower and time, and improve the efficiency and accuracy of data cleaning. Attached Figure Description

[0018] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a training data processing method provided in an embodiment of this application;

[0020] Figure 2 This is a flowchart illustrating a classification and identification method provided in an embodiment of this application;

[0021] Figure 3 This is a flowchart illustrating a method for determining associated information provided in an embodiment of this application;

[0022] Figure 4 This is a schematic flowchart of a target image acquisition method provided in an embodiment of this application;

[0023] Figure 5 This is a flowchart illustrating another method for obtaining a target image provided in an embodiment of this application;

[0024] Figure 6 This is a flowchart illustrating a classification and recognition model training method provided in an embodiment of this application;

[0025] Figure 7 This is a flowchart illustrating a data cleaning model training method provided in an embodiment of this application;

[0026] Figure 8 This is a schematic diagram of the structure of a training data processing device provided in an embodiment of this application;

[0027] Figure 9 This is a hardware structure block diagram of a server for a training data processing method provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0030] The following describes a training data processing method according to this application. Figure 1 This is a flowchart illustrating a training data processing method provided in an embodiment of this application. This application provides the operational steps of the method described in the embodiment or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiment is merely one possible execution order among many and does not represent the only execution order. In actual system or server product execution, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiment or drawings. Specifically, as... Figure 1 As shown, the method may include:

[0031] S101: Obtain at least two images with the same classification label identified by the first classification recognition model, as well as the image feature information and confidence level corresponding to the at least two images.

[0032] In this embodiment of the application, the first classification and recognition model can identify the target objects in the at least two images, and determine the classification identifiers corresponding to the at least two images based on the categories of the target objects in the at least two images. In practical applications, target objects can include people, objects, and scenes, such as doctors, animals, plants, and buildings.

[0033] In this embodiment, the classification identifiers corresponding to the at least two images can characterize the category information predicted by the first classification and recognition model for the at least two images. Specifically, when the target objects of the at least two images include people, the classification identifiers corresponding to the at least two images can characterize the occupation category predicted by the first classification and recognition model for the at least two images, such as "doctor". When the target objects of the at least two images include objects, the classification identifiers corresponding to the at least two images can characterize the object category predicted by the first classification and recognition model for the at least two images, such as "apple". When the target objects of the at least two images include scenery, the classification identifiers corresponding to the at least two images can characterize the scenery category predicted by the first classification and recognition model for the at least two images, such as "river".

[0034] In this embodiment of the application, the image feature information corresponding to the at least two images may include coarse-grained features and fine-grained features corresponding to the at least two images. The coarse-grained features may include, but are not limited to, the shape and color features of the target object, such as edge features and color saturation; the fine-grained features may include, but are not limited to, texture features and spatial relationship features, such as gray-level co-occurrence matrix and angle.

[0035] In this embodiment, the confidence level corresponding to the at least two images can characterize the probability that the category information predicted by the first classification and recognition model for the at least two images is the same as the target category. Specifically, the target category can be the true category of the at least two images.

[0036] In an optional embodiment, such as Figure 2 As shown, the above method also includes:

[0037] S201: Obtain at least two images to be classified.

[0038] In this embodiment of the application, the at least two images to be classified can be images that need to be classified and identified.

[0039] S203: Using the first classification and recognition model, feature extraction is performed on the at least two images to be classified to obtain image feature information corresponding to the at least two images to be classified.

[0040] In this embodiment of the application, the first classification and recognition model includes an image feature extraction network. By inputting the at least two images to be classified into the image feature extraction network, image feature information corresponding to the at least two images to be classified can be obtained.

[0041] In this embodiment of the application, the image feature extraction network can be a convolutional neural network. The type of the convolutional neural network may include, but is not limited to, ResNet (Residual Network), DenSENet (Dense Convolutional Network), and SENet (Squeeze-and-Excitation Networks).

[0042] S205: Classify and identify the image feature information corresponding to the at least two images to be classified, and obtain the classification labels and confidence levels corresponding to the at least two images to be classified.

[0043] In this embodiment of the application, the first classification and recognition model further includes a classification and recognition network. By inputting the image feature information corresponding to the at least two images to be classified into the classification and recognition network, the classification labels and confidence levels corresponding to the at least two images to be classified can be obtained.

[0044] In this embodiment of the application, the classification and recognition network can be a fully connected network. There is no upper limit to the number of layers in the fully connected network, and it can be set according to the needs of the actual application.

[0045] Accordingly, obtaining at least two images with the same classification label identified by the first classification recognition model, and the image feature information and confidence level corresponding to the at least two images, includes:

[0046] S207: Identify at least two images with the same classification label from at least two images to be classified.

[0047] S209: Obtain the image feature information corresponding to the at least two images from the image feature information corresponding to the at least two images to be classified.

[0048] S211: Obtain the confidence scores corresponding to the at least two images from the confidence scores corresponding to the at least two images to be classified.

[0049] In this embodiment of the application, by obtaining at least two images with the same classification identifier, as well as the image feature information and confidence level corresponding to the at least two images, data support can be provided for subsequent data cleaning operations.

[0050] S103: Determine the association information of the at least two images based on the image feature information and confidence level corresponding to the at least two images; wherein the association information characterizes the correlation between the at least two images.

[0051] In this embodiment of the application, the apparent similarity between the at least two images can be determined based on the image feature information corresponding to the at least two images.

[0052] The more similar the image feature information of any two of the at least two images, the higher the apparent similarity between the two images.

[0053] In this embodiment of the application, the degree of category similarity between the at least two images can be determined based on the confidence levels corresponding to the at least two images.

[0054] If the confidence levels of any two of the at least two images are more similar, it indicates that the confidence levels of the category information corresponding to the two images are similar.

[0055] If the confidence level of any two of the at least two images is higher and they are more similar, it indicates that the category similarity of the two images is higher and that they are both images with correct classification labels. Otherwise, it indicates that the category similarity of the two images is lower and that they are images with different classification labels. For example, one image has a correct classification label and the other image has an incorrect classification label.

[0056] In this embodiment of the application, the association information of the at least two images is determined by combining the image feature information and confidence level corresponding to the at least two images. The association information can be used to comprehensively represent the appearance similarity and category similarity of the at least two images, and data cleaning is performed on the at least two images to ensure the effectiveness and accuracy of data cleaning.

[0057] In one specific embodiment, in order to determine the association information of the at least two images, such as Figure 3 As shown, determining the association information of the at least two images based on their corresponding image feature information and confidence level includes:

[0058] S301: Determine the image feature association information of the at least two images based on the image feature information corresponding to the at least two images.

[0059] In this embodiment of the application, the image feature association information of the at least two images includes the apparent similarity value of any two images among the at least two images, which can characterize the degree of apparent similarity between any two images among the at least two images.

[0060] Specifically, taking the i-th and j-th images out of at least two images as an example, the more similar the image feature information of the i-th and j-th images, the higher their apparent similarity.

[0061] In practical applications, the image feature information corresponding to the at least two images can be represented in the form of multidimensional vectors to obtain image feature vectors corresponding to the at least two images. By calculating the cosine distance or Euclidean distance between the image feature vectors corresponding to any two of the at least two images, an apparent similarity value that can characterize the apparent similarity between any two of the at least two images is obtained.

[0062] In practical applications, the apparent similarity value of each pair of images can be represented in matrix form to obtain the image feature association matrix. It can be understood that the image feature association matrix is ​​image feature association information represented in matrix form. When there are N images and the dimension of the image feature vector corresponding to each image is N, the image feature association matrix is ​​an N*N dimensional matrix.

[0063] It can be understood that the image feature association matrix is ​​image feature association information in matrix form.

[0064] S303: Determine the confidence association information of the at least two images based on the confidence levels corresponding to the at least two images.

[0065] In this embodiment of the application, the confidence association information of the at least two images includes the category similarity value corresponding to any two of the at least two images, which can characterize the category similarity of any two of the at least two images.

[0066] Specifically, taking the i-th and j-th images out of at least two images as an example, the smaller the difference in confidence scores between the i-th and j-th images, the more likely they are images of the same category. If the difference in confidence scores between the i-th and j-th images is small, and their respective confidence scores are both large, then they are more likely to be images of the same category and correctly classified.

[0067] In practical applications, a category similarity value that can characterize the degree of category similarity between any two images can be calculated based on the confidence level of any two images among the at least two images. Based on the category similarity value of each pair of images, the confidence association information of the at least two images can be obtained.

[0068] Taking the i-th and j-th images out of at least two images as an example, their category similarity value C ij C can be calculated using the following formula: ij =1-|c i -c j |. Where c i Let c be the confidence level corresponding to the i-th image. j Let be the confidence score corresponding to the j-th image.

[0069] In this embodiment, the category similarity value of each pair of images can be represented in matrix form to obtain a confidence correlation matrix. When the number of images is N, the confidence correlation matrix is ​​an N*N dimensional matrix.

[0070] It can be understood that this confidence correlation matrix is ​​confidence correlation information in matrix form.

[0071] S305: Determine the association information of the at least two images based on the image feature association information and the confidence association information.

[0072] In practical applications, the product of the above image feature correlation matrix and the above confidence correlation matrix can be calculated to obtain the correlation matrix S.

[0073] It can be understood that S is an N*N dimensional matrix.

[0074] It can be understood that the association matrix is ​​a matrix of association information, which may include the combined similarity value of each pair of images mentioned above.

[0075] Taking the i-th and j-th images from at least two images as an example, their combined similarity value S ij etc. F ij With C ij The accumulation of.

[0076] In this embodiment, the apparent similarity and category similarity between at least two images can be represented by determining the association information of at least two images. This is beneficial for the subsequent data cleaning process, where the data cleaning model can clean out images with high apparent similarity and high category similarity as images with correct classification labels, and identify the remaining images as images with incorrect classification labels.

[0077] S105: Input the image feature information and the association information corresponding to the at least two images into the data cleaning model for data cleaning to obtain the target image with incorrect classification label in the at least two images; wherein, the data cleaning model is trained based on the second training data with the same sample classification label identified by the first classification recognition model, and the sample feature information and sample confidence corresponding to the second training data; wherein, the target image is used to train the second classification recognition model.

[0078] In this embodiment, the data cleaning model includes an association feature extraction network and an identification network.

[0079] In a specific embodiment, Figure 4As shown, the step of inputting the image feature information and the association information corresponding to the at least two images into the data cleaning model for data cleaning to obtain the target image with incorrect classification in the at least two images includes:

[0080] S401: Using the data cleaning model, feature extraction is performed on the image feature information and the association information corresponding to the at least two images to obtain the association feature information corresponding to the at least two images.

[0081] In this embodiment, the associated feature information corresponding to each image includes associated feature values ​​corresponding to at least two associated features. These at least two associated features can characterize the correlation between the corresponding image and at least two other images, and the associated feature values ​​corresponding to these two associated features can be the weights of these at least two associated features, characterizing the degree of correlation between the corresponding image and at least two other images.

[0082] In an optional embodiment, the data cleaning model includes an association feature extraction network, which can input the image feature information corresponding to the at least two images and the association information into the association feature extraction network to obtain the association feature information corresponding to the at least two images.

[0083] In this embodiment of the application, the associated feature extraction network includes at least two convolutional layers.

[0084] In this embodiment of the application, the image feature information corresponding to the at least two images is used as the input parameter of the input layer in the at least two convolutional layers, and the image feature information corresponding to the at least two images is used to determine the AGG (Aggregation) between the nodes of each convolutional layer.

[0085] In this context, the image feature information corresponding to each image is used as the input parameter for the corresponding node in the input layer.

[0086] Specifically, by determining AGG based on the image feature information corresponding to at least two images, the correlation between nodes can be constructed, so that the correlation between nodes corresponds to the correlation between at least two images.

[0087] In an optional embodiment, mean aggregation can be performed on the association information of at least two images to obtain AGG. See Formula 1 for details:

[0088]

[0089] Here, D is a diagonal matrix of S. The diagonal similarity value of each row in D can be obtained by summing the comprehensive similarity values ​​of each row in S.

[0090] In an optional embodiment, the activation function sigma() can also be used to... Activate the AGG to obtain it. See Formula 2 for details:

[0091]

[0092] Where M and b are learnable parameters during the training phase of the data cleaning model.

[0093] In the above embodiments, Formula 1, compared to Formula 2, has a better aggregation effect on the correlation between nodes and can accelerate the convergence of the image feature extraction network; however, it has a higher computational complexity. In practical applications, the formula for calculating AGG can be selected according to actual needs.

[0094] In this embodiment of the application, the calculation formula for AGG can also take many other forms, not limited to the two examples mentioned above.

[0095] In this embodiment of the application, the output parameter Layer_out(L) of the Lth convolutional layer can be calculated using the following formula 3:

[0096] Layer_out(L)=sigma[G(Layer_in(L))·w L ] (Formula 3);

[0097] Where Layer_in(L) represents the input parameters of the Lth convolutional layer, w L sigma() is the learnable parameter during the training phase of the data cleaning model, and is the activation function for each convolutional layer.

[0098] It can be understood that Layer_out(L) is the transitional correlation feature information corresponding to each image obtained by further refining Layer_in(L) by the Lth convolutional layer.

[0099] In an optional embodiment, G(Layer_in(L)) can be calculated using the following formula 4:

[0100] G(Layer_in(L))=Layer_in(L)||(AGG·Layer_in(L)) (Formula 4);

[0101] Here, "||" indicates matrix concatenation of Layer_in(L) and AGG·Layer_in(L).

[0102] In an optional embodiment, the above G(Layer_in(L)) can also be calculated using the following formula 5:

[0103] AGG·Layer_in(L) (Formula 5).

[0104] In an optional embodiment, the above G(Layer_in(L)) can also be calculated using Equation 6:

[0105] Layer_in(L)||(AGG·Layer_in(L)·U) (Formula 6);

[0106] Where U represents the learnable parameters during the training phase of the data cleaning model.

[0107] In an optional embodiment, the above G(Layer_in(L)) can also be calculated using Equation 7:

[0108] Layer_in(L)||(AGG·Layer_in(L)·U) (Formula 7).

[0109] The above examples illustrate four formulas for calculating G(Layer_in(L)), each with different computational complexities and aggregation effects on inter-node correlations. In practical applications, the formula for calculating G(Layer_in(L)) can be selected based on actual needs.

[0110] In this embodiment of the application, the calculation formula for G(Layer_in(L)) can also have more forms, not limited to the four examples mentioned above.

[0111] In this embodiment of the application, there is no upper limit to the number of convolutional layers.

[0112] In practical applications, considering the processing efficiency and effect of the associated feature extraction network, the number of convolutional layers is preferably four.

[0113] In this embodiment of the application, the association features of each of the at least two images and other images are extracted by the association feature network to obtain the association feature information corresponding to the at least two images. This can ensure the accuracy and effectiveness of the association feature information corresponding to the at least two images and improve the efficiency and effect of data cleaning.

[0114] S403: Based on the associated feature information corresponding to the at least two images, obtain the target image with incorrect classification labeling in the at least two images.

[0115] In an optional embodiment, the data cleaning model further includes an identification network, which can input the associated feature information corresponding to the at least two images to obtain the target image with incorrect classification in the at least two images.

[0116] In this embodiment of the application, the discrimination network can be a fully connected neural network.

[0117] In one specific embodiment, in order to identify the target image with an incorrect classification label among the at least two images, such as Figure 5 As shown, obtaining the target image with incorrect classification labeling from the at least two images based on the associated feature information corresponding to the at least two images includes:

[0118] S501: Weight the associated feature values ​​in the associated feature information corresponding to the at least two images to obtain the target feature values ​​corresponding to the at least two images.

[0119] In this embodiment of the application, the weighted processing method can be weighted summation calculation.

[0120] In this embodiment of the application, the weighted summation of the associated feature values ​​in the associated feature information corresponding to the at least two images can be performed to obtain the average value (target feature value) of the associated feature values ​​corresponding to the at least two associated features. The average value is used to reasonably measure the comprehensive correlation between the corresponding image and other images.

[0121] For example, the associated feature information corresponding to image A includes the associated feature values ​​of image A with images B, C, and D, which are 1, 3, 2, 1, and 2, respectively. With the weights of the aforementioned associated features being 1, 2, 3, 3, and 1, the weighted summation of the associated feature values ​​yields a target feature value of 3.6 for image A.

[0122] S503: Normalize the target feature values ​​corresponding to the at least two images to obtain the accuracy of the classification labels corresponding to the at least two images.

[0123] In this embodiment, an activation function can be used to activate the target feature values ​​corresponding to at least two images to achieve normalization of the target feature values ​​corresponding to at least two images. The activation function can include commonly used activation functions such as the sigmoid function.

[0124] For example, by using the sigmoid function to normalize the target feature value 3.6 of image A in S501, the classification accuracy of image A is 0.97.

[0125] By normalizing the target feature values ​​corresponding to at least two images, the target feature values ​​corresponding to at least two images can be mapped to the range of 0 to 1 to meet the numerical range requirements of probability. The target feature values ​​corresponding to at least two images are transformed into the accuracy of the classification labels corresponding to at least two images, which is helpful to quickly distinguish between images with correct classification labels and images with incorrect classification labels from at least two images.

[0126] S505: The image whose corresponding classification label accuracy is lower than a preset threshold is taken as the target image with the incorrect classification label among the at least two images.

[0127] In this embodiment, the preset threshold can be set to any value between 0.4 and 0.6, depending on the actual application requirements. In this embodiment, considering data balance, the preset threshold is preferably 0.5.

[0128] Taking the aforementioned image A as an example, since the accuracy of the classification label corresponding to image A is 0.97, which is higher than the preset threshold of 0.5, it is the target image with the correct classification label among at least two images.

[0129] In this embodiment of the application, by processing the associated feature information corresponding to the at least two images through the discrimination network, the automatic discrimination of the classification labels corresponding to the at least two images can be realized, further improving the automation of training data processing.

[0130] In an optional embodiment, the second classification recognition model can be the first classification recognition model.

[0131] Accordingly, in an optional embodiment, the above method further includes:

[0132] Obtain the first training data of the first classification and recognition model;

[0133] The target image is used as a negative sample to supplement the first training data to obtain the target training data of the first classification and recognition model.

[0134] In this embodiment of the application, the first training data may be the original training set of the first classification and recognition model.

[0135] In this embodiment of the application, the target training data can be a new training set obtained by supplementing the original training set of the first classification and recognition model.

[0136] Accordingly, in an alternative embodiment, such as Figure 6 As shown, the above method also includes:

[0137] S601: Input the target training data into the first classification and recognition model for classification and recognition, and obtain the predicted classification label corresponding to the target training data.

[0138] S603: Obtain the target classification identifier corresponding to the target training data.

[0139] S605: Calculate the first target loss of the first classification and recognition model based on the predicted classification identifier and the target classification identifier corresponding to the target training data.

[0140] S607: If the first target loss does not meet the first preset condition, adjust the first model parameters of the first classification and recognition model according to the first target loss until the first target loss meets the first preset condition, and obtain the first target classification and recognition model trained based on the target training data.

[0141] In S603, the target classification identifier represents the true classification of the corresponding target training data.

[0142] In S605, the loss function used to calculate the loss of the first target may include, but is not limited to, logistic regression loss function, weighted cross-entropy loss function, cosine loss function, and intermediate loss function.

[0143] In S607, the first model parameters include the network parameters of the image feature extraction network and the classification network.

[0144] In S607, the first target loss satisfying the first preset condition can be that the first target loss is less than or equal to a specified threshold, or the difference between the first target loss corresponding to two consecutive training processes and the first target loss corresponding to the previous training is less than a certain threshold. In this embodiment, the specified threshold and the certain threshold can be set according to actual training needs.

[0145] In practical applications, during multiple iterations of training the first classification recognition model, a portion of training samples are randomly selected from the target training data each time to participate in the training. Correspondingly, updating the second target loss based on the updated first classification recognition model may include randomly selecting a portion of training samples from the target training data and repeating the steps S601 to S605 above to determine the second target loss in conjunction with the updated first classification recognition model.

[0146] In this embodiment, by adding incorrectly classified target images identified by the first classification recognition model as negative samples to the original training set of the first classification recognition model, a new training set is obtained. Training the first classification recognition model using this new training set improves its classification ability. This automatic and rapid method of selecting positive and negative samples greatly reduces the manual work involved in selecting them, providing cleaner training data, accelerating model iteration, and improving model performance.

[0147] In an optional embodiment, such as Figure 7 As shown, the above method also includes:

[0148] S701: Obtain second training data that has the same sample classification identifier as the first classification recognition model, as well as sample feature information and sample confidence level corresponding to the second training data.

[0149] In this embodiment of the application, the second training data includes at least two sample images. In this embodiment of the application, the second training data is different from the first training data.

[0150] In this embodiment, the first classification and recognition model can identify the target objects in the at least two sample images, and determine the sample classification identifier corresponding to the second training data based on the categories of the target objects in the at least two sample images. Specifically, the sample classification identifier corresponding to the at least two sample images can characterize the category information corresponding to the at least two sample images predicted by the first classification and recognition model.

[0151] In an optional embodiment, the above method further includes:

[0152] At least two sample images to be classified are obtained, and the at least two sample images to be classified are input into the first classification and recognition model to obtain the sample classification identifiers of the at least two sample images to be classified, as well as the sample feature information and sample confidence of the at least two sample images to be classified.

[0153] In this embodiment, the first classification and recognition model includes an image feature extraction network and a classification and recognition network. Accordingly, the process of inputting at least two sample images to be classified into the first classification and recognition model to obtain sample classification identifiers for at least two sample images, as well as sample feature information and sample confidence levels corresponding to the at least two sample images includes:

[0154] Input at least two sample images to be classified into an image feature extraction network to obtain sample feature information corresponding to at least two sample images to be classified.

[0155] Input the sample feature information corresponding to at least two sample images to be classified into the classification and recognition model to obtain the sample classification label and sample confidence level corresponding to at least two sample images to be classified.

[0156] Accordingly, the above-mentioned acquisition of second training data that has the same sample classification label as identified by the first classification recognition model, as well as sample feature information and sample confidence corresponding to the second training data, includes:

[0157] Based on the sample classification labels corresponding to at least two sample images to be classified, determine the sample images to be classified with the same sample classification label from at least two sample images to be classified, and obtain the second training data;

[0158] The sample feature information and sample confidence level corresponding to the second training data are determined from the sample feature information and sample confidence level corresponding to at least two sample images to be classified.

[0159] S703: Determine the sample association information of the second training data based on the sample feature information and sample confidence level corresponding to the second training data; wherein, the sample association information characterizes the correlation between the second training data.

[0160] In this embodiment of the application, the sample feature information corresponding to the second training data may include coarse-grained features and fine-grained features corresponding to the at least two sample images.

[0161] In this embodiment, the confidence level corresponding to the second training data can characterize the probability that the sample category information corresponding to at least two sample images predicted by the first classification and recognition model is the same as the target sample category. Specifically, the target sample category can be the true category of the at least two sample images.

[0162] In this embodiment of the application, the sample image feature association information of the second training data can be determined based on the sample feature information corresponding to the second training data. For details on how to obtain the aforementioned sample image feature association information, please refer to the foregoing description of how to determine the image feature association information of at least two images based on the image feature information corresponding to at least two images; further details will not be repeated here.

[0163] In this embodiment of the application, the sample confidence correlation information of the second training data can be determined based on the sample confidence corresponding to the second training data. For details on how to obtain the aforementioned sample confidence correlation information, please refer to the foregoing description of how to determine the confidence correlation information of at least two images based on their corresponding confidence levels; further details will not be repeated here.

[0164] In this embodiment, the sample association information of the second training data can be determined based on the sample feature information and sample confidence level corresponding to the second training data. For details on how to obtain the aforementioned sample association information, please refer to the foregoing description of how to determine the association information of at least two images based on the image feature association information and confidence level association information corresponding to at least two images; further details will not be repeated here.

[0165] S705: Input the sample feature information and the sample association information corresponding to the second training data into the initial data cleaning model for data cleaning to obtain the identification result of the sample classification identifier corresponding to the second training data.

[0166] In this embodiment, a data cleaning model is used to extract features from the sample feature information and sample association information corresponding to at least two sample images in the second training data, thereby obtaining sample association feature information corresponding to at least two sample images. Based on the sample association feature information corresponding to at least two sample images, an identification result for the classification labels corresponding to at least two sample images is obtained. The identification result can characterize whether the classification labels corresponding to at least two sample images are correct or incorrect.

[0167] In this embodiment, the sample association feature information includes sample association feature values ​​corresponding to at least two sample association features. These at least two sample association features can characterize the correlation between the corresponding sample image and at least two other sample images. The sample association feature values ​​corresponding to these two sample association features can be the weights of these at least two sample association features, and can characterize the degree of correlation between the corresponding sample image and at least two other sample images.

[0168] In an optional embodiment, the initial data cleaning model includes an association feature extraction network and an identification network.

[0169] In an optional embodiment, the above-mentioned data cleaning model is used to extract features from the sample feature information and the sample association information corresponding to at least two sample images of the second training data, resulting in sample association feature information corresponding to at least two sample images, including:

[0170] The sample feature information corresponding to at least two sample images and the above-mentioned sample association information are input into the association feature extraction network to obtain the sample association feature information corresponding to the second training data.

[0171] In an optional embodiment, the above-mentioned identification result of the classification identifier corresponding to at least two sample images based on the sample association feature information corresponding to at least two sample images includes:

[0172] The sample association feature information corresponding to at least two sample images is input into the discrimination network. The discrimination network performs weighted processing on the sample association feature values ​​in the sample association feature information corresponding to at least two sample images to obtain the sample target feature values ​​corresponding to at least two sample images.

[0173] Normalize the target feature values ​​corresponding to at least two sample images to obtain the accuracy of sample classification labels corresponding to at least two sample images;

[0174] Based on the accuracy of the sample classification labels corresponding to at least two sample images, determine the identification result of the classification labels corresponding to at least two sample images.

[0175] S707: Obtain the sample target classification identifier corresponding to the second training data.

[0176] In this embodiment of the application, the sample target classification identifier corresponding to the second training data can represent the true category corresponding to the second training data.

[0177] S709: Calculate the second target loss of the initial data cleaning model based on the identification results of the sample classification identifiers corresponding to the second training data and the sample target classification identifiers.

[0178] In this embodiment of the application, the loss function used to calculate the loss of the second target may include, but is not limited to, the cross-entropy function.

[0179] S711: If the second target loss does not meet the second preset condition, update the second model parameters of the initial data cleaning model, update the second target loss based on the updated initial data cleaning model until the second target loss meets the second preset condition, and use the currently updated initial data cleaning model as the data cleaning model.

[0180] In this embodiment of the application, the second model parameters include the network parameters of the feature extraction network and the discrimination network.

[0181] In this embodiment, the second target loss satisfies the second preset condition by being less than or equal to a specified threshold, or by the difference between the second target loss corresponding to two training processes and the second target loss corresponding to the previous training learning being less than a certain threshold. In this embodiment, the specified threshold and the certain threshold can be set according to actual training needs.

[0182] In practical applications, during multiple iterations of training the first classification and recognition model, a portion of training samples are randomly selected from the second training data each time to participate in the training. Correspondingly, updating the second target loss based on the updated data can include randomly selecting a portion of training samples from the second training data and repeating the steps S701 to S709 above to determine the second target loss in combination with the updated data.

[0183] In this embodiment, an initial data cleaning model is trained to obtain a data cleaning model, and the data cleaning model is used to automate data cleaning, thereby further improving the automation of training data processing.

[0184] In this embodiment, a data cleaning model is trained to clean at least two images with the same classification label identified by the first classification recognition model, obtaining a target image with an incorrect classification label from at least two images. This target image is then used to train the second classification recognition model, or it can be used as a negative sample to supplement the training data of the first classification recognition model. This automates the training data processing and automatically obtains supplementary training data. By using the target image to train the first classification recognition model, the classification recognition capability of the first classification recognition model can be effectively improved.

[0185] This application also provides a training data processing apparatus, such as... Figure 8 As shown, the device includes:

[0186] The acquisition module 801 is used to acquire at least two images with the same classification label identified by the first classification recognition model, as well as the image feature information and confidence level corresponding to the at least two images;

[0187] The determining module 803 is used to determine the association information of the at least two images based on the image feature information and confidence level corresponding to the at least two images; wherein the association information characterizes the correlation between the at least two images;

[0188] The data cleaning module 805 is used to input the image feature information and the association information corresponding to the at least two images into the data cleaning model for data cleaning to obtain the target image with incorrect classification label in the at least two images; wherein, the data cleaning model is trained based on the second training data with the same sample classification label identified by the first classification recognition model, and the sample feature information and sample confidence corresponding to the second training data; wherein, the target image is used to train the second classification recognition model.

[0189] In some embodiments, the data cleaning module 805 further includes:

[0190] The associated feature extraction unit is used to extract features from the image feature information and the associated information corresponding to the at least two images through the data cleaning model, so as to obtain the associated feature information corresponding to the at least two images;

[0191] The identification unit is used to obtain the target image with incorrect classification in the at least two images based on the associated feature information corresponding to the at least two images.

[0192] In some embodiments, the authentication unit includes:

[0193] The weighted processing subunit is used to perform weighted processing on the associated feature values ​​in the associated feature information corresponding to the at least two images to obtain the target feature values ​​corresponding to the at least two images.

[0194] The normalization processing subunit is used to normalize the target feature values ​​corresponding to the at least two images to obtain the accuracy of the classification labels corresponding to the at least two images.

[0195] A subunit is determined to identify images whose corresponding classification accuracy is lower than a preset threshold as target images with incorrect classification among the at least two images.

[0196] In some embodiments, the determining module 803 includes:

[0197] The first determining unit is configured to determine the image feature association information of the at least two images based on the image feature information corresponding to the at least two images;

[0198] The second determining unit is used to determine the confidence association information of the at least two images based on the confidence levels corresponding to the at least two images;

[0199] The third determining unit is used to determine the association information of the at least two images based on the image feature association information and the confidence association information.

[0200] In some embodiments, the apparatus further includes:

[0201] The second acquisition module is used to acquire at least two images to be classified.

[0202] The second determining module is used to extract features from the at least two images to be classified using the first classification and recognition model to obtain image feature information corresponding to the at least two images to be classified.

[0203] The third determining module is used to classify and identify the image feature information corresponding to the at least two images to be classified, and to obtain the classification identifier and confidence level corresponding to the at least two images to be classified.

[0204] In an optional embodiment, the apparatus further includes:

[0205] The third acquisition module is used to acquire the first training data of the first classification and recognition model;

[0206] The supplementary module is used to supplement the first training data with the target image as a negative sample to obtain the target training data of the first classification and recognition model.

[0207] In some embodiments, the apparatus further includes:

[0208] The classification and recognition module is used to input the target training data into the first classification and recognition model for classification and recognition, and to obtain the predicted classification label corresponding to the target training data.

[0209] The fourth acquisition module is used to acquire the target classification identifier corresponding to the target training data;

[0210] The first calculation module is used to calculate the first target loss of the first classification and recognition model based on the predicted classification label and the target classification label corresponding to the target training data.

[0211] The first training module is used to adjust the first model parameters of the first classification and recognition model according to the first target loss when the first target loss does not meet the first preset condition, until the first target loss meets the first preset condition, so as to obtain the first target classification and recognition model trained based on the target training data.

[0212] In some embodiments, the apparatus further includes:

[0213] The acquisition module is also used to acquire second training data that has the same sample classification identifier as the first classification recognition model, as well as sample feature information and sample confidence corresponding to the second training data;

[0214] The determining module is further configured to determine the sample association information of the second training data based on the sample feature information and sample confidence level corresponding to the second training data; wherein, the sample association information characterizes the correlation between the second training data.

[0215] The data cleaning module is also used to input the sample feature information and the sample association information corresponding to the second training data into the initial data cleaning model for data cleaning, so as to obtain the identification result of the sample classification identifier corresponding to the second training data;

[0216] The fourth acquisition module is also used to acquire the sample target classification identifier corresponding to the second training data;

[0217] The second calculation module is used to calculate the second target loss of the initial data cleaning model based on the identification result of the sample classification identifier corresponding to the second training data and the sample target classification identifier.

[0218] The second training module is used to update the second model parameters of the initial data cleaning model when the second target loss does not meet the second preset condition, update the second target loss based on the updated data cleaning model until the second target loss meets the second preset condition, and use the currently updated initial data cleaning model as the data cleaning model.

[0219] The apparatus and method embodiments described herein are based on the same inventive concept.

[0220] This application embodiment also provides a training data processing device, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the training data processing method as described above.

[0221] The apparatus and method embodiments described in the equipment examples are based on the same inventive concept.

[0222] This application also provides a computer-readable storage medium storing at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the training data processing method described above.

[0223] The computer-readable storage medium and method embodiments described herein are based on the same inventive concept.

[0224] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations described above.

[0225] This application provides a training data processing server, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the training data processing method provided in the above method embodiments.

[0226] The memory can be used to store software programs and modules. The processor executes various functional applications and training data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0227] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, servers, or similar computing devices. Taking running on a server as an example, Figure 9 This is a hardware structure block diagram of a server for a training data processing method provided in an embodiment of this application. For example... Figure 9 As shown, the server 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 910 (CPUs 910 may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory 930 for storing data, and one or more storage media 920 (e.g., one or more mass storage devices) for storing application programs 923 or data 922. The memory 930 and storage media 920 may be temporary or persistent storage. The program stored in the storage media 920 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 910 may be configured to communicate with the storage media 920 and execute the series of instruction operations stored in the storage media 920 on the server 900. Server 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input / output interfaces 940, and / or one or more operating systems 921, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0228] The input / output interface 940 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 900. In one example, the input / output interface 940 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 940 may be a radio frequency (RF) module for wireless communication with the Internet.

[0229] Those skilled in the art will understand that Figure 9 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 900 may also include... Figure 9 The more or fewer components shown, or having the same Figure 9 The different configurations shown.

[0230] The embodiments of this application also provide a storage medium, which can be disposed in a server to store at least one instruction, at least one program, code set or instruction set related to implementing a training data processing method in the method embodiments. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the training data processing method provided in the above method embodiments.

[0231] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0232] As can be seen from the embodiments of the training data processing method, apparatus, server or storage medium provided in this application, this application uses a data cleaning model to clean at least two images with the same classification label identified by the first classification recognition model to obtain a target image with an incorrect classification label. The target image is then used to train the second classification recognition model. Alternatively, the target image can be used as a negative sample to supplement the training data of the first classification recognition model, thereby automating the training data processing and automatically obtaining the target training data of the first classification recognition model.

[0233] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0234] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0235] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0236] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A training data processing method, characterized in that, The method includes: Obtain at least two images with the same classification label identified by the first classification recognition model, as well as the image feature information and confidence level corresponding to the at least two images; Based on the image feature information and confidence level corresponding to the at least two images, the association information of the at least two images is determined; wherein, the association information characterizes the correlation between the at least two images; The image feature information and the association information corresponding to the at least two images are input into the data cleaning model for data cleaning to obtain the target image with incorrect classification label in the at least two images; wherein, the data cleaning model is trained based on the second training data with the same sample classification label identified by the first classification recognition model, and the sample feature information and sample confidence corresponding to the second training data; wherein, the target image is used to train the second classification recognition model.

2. The method according to claim 1, characterized in that, The step of inputting the image feature information and the association information corresponding to the at least two images into the data cleaning model for data cleaning to obtain the target image with incorrect classification in the at least two images includes: Using the data cleaning model, feature extraction is performed on the image feature information and the association information corresponding to the at least two images to obtain the association feature information corresponding to the at least two images. Based on the associated feature information corresponding to the at least two images, the target image with incorrect classification labeling in the at least two images is obtained.

3. The method according to claim 2, characterized in that, The step of obtaining the target image with incorrect classification labeling from the at least two images based on the associated feature information corresponding to the at least two images includes: The associated feature values ​​in the associated feature information corresponding to the at least two images are weighted to obtain the target feature values ​​corresponding to the at least two images; The target feature values ​​corresponding to the at least two images are normalized to obtain the accuracy of the classification labels corresponding to the at least two images; Images whose corresponding classification accuracy is lower than a preset threshold are designated as target images with incorrect classification among the at least two images.

4. The method according to claim 1, characterized in that, The step of determining the association information of the at least two images based on the image feature information and confidence level corresponding to the at least two images includes: Based on the image feature information corresponding to the at least two images, determine the image feature association information of the at least two images; Based on the confidence scores corresponding to the at least two images, determine the confidence score association information of the at least two images; The association information of the at least two images is determined based on the image feature association information and the confidence association information.

5. The method according to claim 1, characterized in that, Before obtaining at least two images with the same classification label identified by the first classification recognition model, and the image feature information and confidence scores corresponding to the at least two images, the method further includes: Obtain at least two images to be classified; The first classification and recognition model is used to extract features from the at least two images to be classified, thereby obtaining image feature information corresponding to the at least two images to be classified. The image feature information corresponding to the at least two images to be classified is classified and identified to obtain the classification labels and confidence scores corresponding to the at least two images to be classified.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain the first training data of the first classification and recognition model; The target image is used as a negative sample to supplement the first training data to obtain the target training data of the first classification and recognition model.

7. The method according to claim 6, characterized in that, The method further includes: The target training data is input into the first classification and recognition model for classification and recognition to obtain the predicted classification label corresponding to the target training data; Obtain the target classification identifier corresponding to the target training data; Based on the predicted classification identifier and the target classification identifier corresponding to the target training data, calculate the first target loss of the first classification and recognition model; If the first target loss does not meet the first preset condition, the first model parameters of the first classification and recognition model are adjusted according to the first target loss until the first target loss meets the first preset condition, thereby obtaining the first target classification and recognition model trained based on the target training data.

8. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain second training data that has the same sample classification label as the first classification recognition model, as well as sample feature information and sample confidence level corresponding to the second training data; Based on the sample feature information and sample confidence level corresponding to the second training data, the sample association information of the second training data is determined; wherein, the sample association information characterizes the correlation between the second training data. The sample feature information and the sample association information corresponding to the second training data are input into the initial data cleaning model for data cleaning to obtain the identification result of the sample classification identifier corresponding to the second training data. Obtain the sample target classification identifier corresponding to the second training data; Based on the identification results of the sample classification identifiers and the sample target classification identifiers corresponding to the second training data, the second target loss of the initial data cleaning model is calculated; If the second target loss does not meet the second preset condition, the second model parameters of the initial data cleaning model are updated, and the second target loss is updated based on the updated initial data cleaning model until the second target loss meets the second preset condition. The current updated initial data cleaning model is then used as the data cleaning model.

9. A training data processing apparatus, characterized in that, The device includes: The first acquisition module is used to acquire at least two images with the same classification label identified by the first classification recognition model, as well as the image feature information and confidence level corresponding to the at least two images; The determining module is configured to determine the association information of the at least two images based on the image feature information and confidence level corresponding to the at least two images; wherein the association information characterizes the correlation between the at least two images; The data cleaning module is used to input the image feature information and the association information corresponding to the at least two images into the data cleaning model for data cleaning to obtain the target image with incorrect classification label in the at least two images; wherein, the data cleaning model is trained based on the second training data with the same sample classification label identified by the first classification recognition model, and the sample feature information and sample confidence corresponding to the second training data; wherein, the target image is used to train the second classification recognition model.

10. A training data processing device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the training data processing method as described in any one of claims 1 to 8.

Images