A method, device, and medium for predicting sample categories in model training

Abstract

The present application relates to the technical field of model training, and particularly relates to a sample category prediction method and device in model training, equipment and medium. The method obtains the category feature and category probability estimation result of each sample, constructs the adjacent graph structure information according to the similarity of the category feature, obtains the category residual of the sample, obtains the modified and smoothed category probability estimation result according to the adjacent graph structure information, the category residual and the category probability estimation result, combines the modified and smoothed operation, determines the category probability prediction result in the current iteration according to the category probability prediction result in the last iteration and the modified and smoothed category probability estimation result in the current iteration, determines the model loss to train the to-be-trained model, and fuses the sample adjacent information and the sample itself information, so that the category probability prediction result fuses the historical iteration information and the mutual relationship between the samples in the feature dimension, and the accuracy of the to-be-trained model is improved.

Description

Technical Field

[0001] This invention relates to the field of model training technology, and in particular to a method, apparatus, device, and medium for predicting sample categories in model training. Background Technology

[0002] The performance of deep learning models depends heavily on massive amounts of high-quality labeled data. However, in real-world scenarios, data is often labeled manually, which can lead to some incorrect labels. In scenarios with noisy labels, classification models often overfit to the noisy labels, resulting in a decrease in the classification accuracy of the model.

[0003] In scenarios with noisy labels, to reduce the impact of noisy labels on training the classification model, an EMA (Estimated Target Estimation) method is proposed. This method iteratively calculates the historical prediction probabilities of the classification model to generate a corresponding estimated target as a soft label for each sample. Regularization and cross-entropy are designed as model losses to reduce the impact of noisy labels on model training. However, this method only considers the cumulative effect of historical prediction results and fails to take into account the interrelationships of feature dimensions between samples. This results in poor accuracy of the generated estimated targets, reducing the accuracy of the classification model after training.

[0004] Therefore, improving the classification accuracy of the model in noisy labeling scenarios has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, apparatus, computer equipment and medium for predicting sample categories during model training, in order to solve the problem of low classification accuracy of existing models in noisy labeled scenarios.

[0006] In a first aspect, embodiments of the present invention provide a method for predicting sample categories during model training, the method comprising:

[0007] When training the model to be trained using the training sample set, the category features and category probability estimation results of each sample in the current iteration are obtained. For any sample in the training sample set, the adjacency graph structure information of the sample is constructed based on the similarity of the category features between the sample and other samples.

[0008] A clean sample set and a non-clean sample set are selected from the training sample set. For any sample in the clean sample set, the class residual of the sample is determined according to the class label of the sample and the corresponding class probability estimation result. The class residuals of all samples in the non-clean sample set are set to a preset value.

[0009] Based on the adjacency graph structure information of all samples, the class residuals, and the class probability estimation results, combined with the correction and smoothing operation, the corrected and smoothed class probability estimation results of the corresponding samples are obtained.

[0010] Based on the class probability prediction results from the previous iteration and the corrected and smoothed class probability estimation results from the current iteration, the class probability prediction results for the current iteration are determined. These predicted results are used to determine the model loss in the current iteration by combining the corrected and smoothed class probability estimation results with the class labels. The model to be trained is then trained based on the model loss to obtain a trained model.

[0011] Secondly, embodiments of the present invention provide a device for predicting sample categories during model training, the device comprising:

[0012] The graph structure information acquisition module is used to acquire the category features and category probability estimation results of each sample in the current iteration when training the model to be trained using the training sample set. For any sample in the training sample set, the adjacency graph structure information of the sample is constructed based on the similarity of the category features between the sample and other samples.

[0013] The category residual determination module is used to select a clean sample set and a non-clean sample set from the training sample set. For any sample in the clean sample set, the category residual of the sample is determined according to the category label of the sample and the corresponding category probability estimation result. The category residual of all samples in the non-clean sample set is set to a preset value.

[0014] The correction and smoothing module is used to obtain the corrected and smoothed class probability estimation result of the corresponding sample by combining the adjacency graph structure information, the class residual, and the class probability estimation result of all samples with the correction and smoothing operation.

[0015] The iterative training module is used to determine the class probability prediction result in the current iteration based on the class probability prediction result in the previous iteration and the corrected and smoothed class probability estimation result in the current iteration. The class probability prediction result in the current iteration is used to determine the model loss in the current iteration by combining the corrected and smoothed class probability estimation result and the class label. The model to be trained is trained according to the model loss to obtain the trained model.

[0016] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for predicting sample categories during model training as described in the first aspect.

[0017] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for predicting sample categories during model training as described in the first aspect.

[0018] The beneficial effects of Embodiment 1 of this invention compared with the prior art are as follows: When training the model to be trained using the training sample set, the category features and category probability estimation results of each sample in the current iteration are obtained. For any sample in the training sample set, the adjacency graph structure information of the sample is constructed based on the similarity of the category features between the sample and other samples. Clean and unclean sample sets are selected from the training sample set. For any sample in the clean sample set, the category residual of the sample is determined based on the sample's category label and the corresponding category probability estimation result. The category residuals of all samples in the unclean sample set are set to preset values. Based on the adjacency graph structure information, category residuals, and category probability estimation results of all samples, combined with a correction and smoothing operation, the corrected and smoothed category probability estimation result of the corresponding sample is obtained. Based on the category probability prediction results in the previous iteration and the current iteration... The corrected and smoothed class probability estimation results are used to determine the class probability prediction results in this iteration. The class probability prediction results in this iteration are used to determine the model loss in this iteration by combining the corrected and smoothed class probability estimation results and the class label. The model to be trained is trained according to the model loss to obtain the trained model. By fusing sample adjacency information and the class residuals of the samples themselves, the class residuals are corrected. Based on the adjacency graph structure information and the corrected class residuals, the class probability estimation results are corrected and smoothed iteratively, which improves the accuracy of the class probability estimation results. This makes the class probability prediction results in this iteration simultaneously integrate the training results of the model to be trained in the historical iteration training process, as well as the mutual relationships between training samples in the feature dimension, which effectively improves the accuracy of the class probability prediction results, and thus improves the accuracy of the model to be trained. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an application environment for a sample category prediction method in model training provided in Embodiment 1 of the present invention;

[0021] Figure 2This is a flowchart illustrating a method for predicting sample categories during model training, provided in Embodiment 1 of the present invention.

[0022] Figure 3 This is a schematic diagram of the structure of a sample category prediction device in model training provided in Embodiment 3 of the present invention;

[0023] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Detailed Implementation

[0024] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0025] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0026] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0028] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0030] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0031] Foundational technologies in artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0032] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0033] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0034] The method for predicting sample categories during model training provided in Embodiment 1 of this invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server. The client includes, but is not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0035] See Figure 2 This is a flowchart illustrating a method for predicting sample categories during model training, as provided in Embodiment 1 of the present invention. This method for predicting sample categories during model training can be applied to… Figure 1 For clients in the model training process, the method for predicting sample categories may include the following steps:

[0036] Step S201: When training the model to be trained using the training sample set, obtain the category features and category probability estimation results of each sample in this iteration. For any sample in the training sample set, construct the adjacency graph structure information of the sample based on the similarity of the category features between the sample and other samples.

[0037] In real-world scenarios, training sample sets are typically labeled using manual annotation methods, which may result in some incorrect labels. The training sample set used in this embodiment is a noisy training sample set.

[0038] When training the model to be trained, the samples in the training sample set can be input into the model to be trained for feature extraction and feature analysis to obtain the category features and category probability estimation results for each sample. The category features are the feature information extracted for each sample, and the category probability estimation results can be used to characterize the probability that the corresponding sample belongs to each category.

[0039] In this embodiment, for any sample in the training sample set, the similarity of the category features between the sample and other samples is calculated, and the adjacency graph structure information of the sample is constructed based on the similarity.

[0040] Specifically, for any sample in the training sample set, the K most similar samples are selected as the K nearest neighbors of that sample based on the similarity of the sample's category features to other samples. If other samples are among the K nearest neighbors of that sample, the relationship between that sample and other samples is a preset value 'a'; if other samples are not among the K nearest neighbors of that sample, the relationship between that sample and other samples is a preset value 'b'. Based on this, the adjacency graph structure information of that sample is constructed. In this embodiment, a = 1, b = 0.

[0041] In one embodiment, the adjacency graph structure information of the samples can be an adjacency graph structure matrix, wherein the row and column dimensions of the adjacency graph structure matrix are consistent with the number of samples, and the value of the i-th row and j-th column of the adjacency graph structure matrix is ​​the relationship value between the i-th sample and the j-th sample. The adjacency graph structure matrix is ​​used to characterize the similarity graph structure relationship between samples.

[0042] In this embodiment, the sample category prediction task can be to classify human postures in sample images in scenarios such as gait analysis, video surveillance, and sports science, with corresponding category labels such as stationary, walking, running, squatting, and jumping; it can also be to classify facial attributes in sample images in scenarios such as product audience analysis and population aging analysis, with corresponding category labels such as male, female, child, youth, middle-aged, and elderly; it can also be to classify human emotions in sample images in scenarios such as teaching evaluation, product sales, and personnel interviews, with corresponding category labels such as happy, nervous, sad, disgusted, and bored; and it can also be to classify room styles in sample images in scenarios such as renting, decorating, and buying a house, with corresponding category labels such as pastoral, minimalist, classical, new Chinese, Mediterranean, and Southeast Asian room styles.

[0043] This embodiment takes the classification of human poses in sample images as an example. The model to be trained is a human pose classification model, and the sample images in the training sample set are human pose sample images including various poses. The category labels are various human pose categories such as stationary, walking, running, squatting, and jumping. Since it is difficult to avoid labeling errors when annotating the category labels, some human pose sample images will be labeled with incorrect human pose categories, making the training sample set in this embodiment a noisy human pose image dataset.

[0044] In this embodiment, when training the model to be trained using a noisy human pose image dataset, the category features and category probability estimation results of each human pose sample image in this iteration are obtained. The category features are the feature information extracted for each human pose sample image, and the category probability estimation results can be used to characterize the probability that the corresponding human pose sample image belongs to each human pose category. Then, for any human pose sample image in the training sample set, the adjacency graph structure information of the human pose sample image is constructed based on the similarity of the category features between the human pose sample image and other human pose sample images.

[0045] Optionally, for any sample in the training sample set, the adjacency graph structure information of the sample is constructed based on the similarity of the class features between the sample and other samples, including:

[0046] For any sample in the training sample set, calculate the similarity of the category features between the sample and other samples;

[0047] Based on similarity, a predetermined number of samples are selected from other samples and identified as similar samples. Samples that are not selected from other samples are identified as dissimilar samples.

[0048] Based on similar samples, the first preset relationship value between samples and similar samples, and the second preset relationship value between dissimilar samples and dissimilar samples, the adjacency graph structure information of the samples is constructed.

[0049] Specifically, for any sample in the training sample set, the similarity of the category features between the sample and other samples is calculated. Then, the similarities are arranged in descending order. A preset number of samples with the highest similarity are selected from the other samples and determined as similar samples of the sample. Samples with lower similarity that are not selected from the other samples are determined as dissimilar samples of the sample.

[0050] Then, the relationship between the sample and similar samples is determined as the first preset relationship value, and the relationship between the sample and dissimilar samples is determined as the second preset relationship value. Based on the first preset relationship value between similar samples and samples and similar samples, and the second preset relationship value between dissimilar samples and dissimilar samples, the adjacency graph structure information of the sample is constructed to characterize the mutual relationship between the sample and other samples in the feature dimension.

[0051] This embodiment determines other samples as similar or dissimilar samples of the sample based on the similarity of category features, and determines a preset relationship value between the sample and the sample. Then, based on the first preset relationship value between similar samples and the sample and the second preset relationship value between dissimilar samples and dissimilar samples, it constructs the adjacency graph structure information of the sample, which effectively represents the mutual relationship between each sample and other samples in the feature dimension.

[0052] The above steps involve obtaining the class features and class probability estimation results of each sample in the current iteration when training the model to be trained using the training sample set, and constructing the adjacency graph structure information of any sample in the training sample set based on the similarity of class features between the sample and other samples. The model to be trained extracts class features from the samples in the training sample set and constructs the adjacency graph structure information of the samples based on the similarity of class features between the sample and other samples, effectively representing the relationship between each sample and other samples in the feature dimension.

[0053] Step S202: Select a clean sample set and a non-clean sample set from the training sample set. For any sample in the clean sample set, determine the class residual of the sample based on the class label of the sample and the corresponding class probability estimation result. Set the class residual of all samples in the non-clean sample set to the preset value.

[0054] In this embodiment, the training sample set is a noisy training sample set. First, a clean sample set and a non-clean sample set are selected from the training sample set, and the residual of each sample in the clean sample set and the non-clean sample set are calculated respectively.

[0055] Specifically, for any sample in the clean sample set, the sample's category label is a category label that is manually assigned after category judgment. The category probability estimation result is the result output by the model after the training model extracts and analyzes the features of the sample, which is used to characterize the probability that the corresponding sample belongs to each category. Therefore, the difference between the sample's category label and the corresponding category probability estimation result can be used as the residual of the sample in the clean sample set.

[0056] To reduce the learning and memorization of noisy labels by the model to be trained, the class residuals of all samples in the non-clean sample set are set to a preset value. In this embodiment, the preset value is 0, so as to reduce the impact of noisy labels on the accuracy of the model to be trained.

[0057] This embodiment takes the classification of human poses in sample images as an example. Clean sample image set and non-clean sample image set are selected from the noisy human pose image dataset. For any human pose sample image in the clean sample image set, the difference between the category label of the human pose sample image and the corresponding category probability estimation result is determined as the category residual of the human pose sample image in the clean sample image set. The category residuals of all human pose sample images in the non-clean sample image set are set to preset values.

[0058] The steps described above—selecting clean and unclean sample sets from the training sample set, determining the class residual for any sample in the clean sample set based on the sample's class label and the corresponding class probability estimation result, and setting the class residuals of all samples in the unclean sample set to preset values—reduce the learning and memorization of noisy labels by the training model, thereby reducing the impact of noisy labels on the accuracy of the training model.

[0059] Step S203: Based on the adjacency graph structure information, class residuals, and class probability estimation results of all samples, and combined with the correction and smoothing operation, the corrected and smoothed class probability estimation results of the corresponding samples are obtained.

[0060] In order to improve the accuracy of the class probability estimation results, this embodiment uses the adjacency graph structure information and class residuals of all samples, and combines correction operations to correct the errors in the class probability estimation results. Then, based on the adjacency graph structure information of all samples, and combined with smoothing operations, the class probability estimation results are smoothed so that the differences in class probability estimation results between adjacent samples are small, and the corrected and smoothed class probability estimation results of the corresponding samples are obtained.

[0061] In the process of correcting and smoothing the class probability estimation results, adjacency graph structure information is used to integrate the inter-sample relationships along the feature dimension into the class residuals of the samples during the correction operation. This results in a residual that integrates the sample adjacency information. This residual is then combined with the sample's own residuals for iterative residual propagation to obtain a corrected class residual matrix. This matrix is ​​used to correct the error in the class probability estimation results, resulting in a corrected class probability estimation result. Simultaneously, during the smoothing operation, the adjacency graph structure information integrates the inter-sample relationships along the feature dimension into the corrected class probability estimation results, resulting in a corrected class probability estimation result that integrates the sample adjacency information. This corrected class probability estimation result is then combined with the sample's own corrected class probability estimation results for smoothing iterations to obtain a corrected and smoothed class probability estimation result.

[0062] This embodiment takes the classification of human poses in sample images as an example. In order to improve the accuracy of the category probability estimation results, this embodiment uses the adjacency graph structure information and category residuals of all human pose sample images, and combines correction operations to correct the errors in the category probability estimation results. Then, based on the adjacency graph structure information of all human pose sample images, and combined with smoothing operations, the category probability estimation results are smoothed so that the differences in category probability estimation results between adjacent human pose sample images are small, and the corrected and smoothed category probability estimation results of the corresponding human pose sample images are obtained.

[0063] Optionally, based on the adjacency graph structure information, class residuals, and class probability estimation results of all samples, combined with a correction and smoothing operation, the corrected and smoothed class probability estimation results for the corresponding samples are obtained, including:

[0064] Based on the adjacency graph structure information of all samples, the class residuals, and the preset residual parameters, combined with the preset residual correction model, the corrected class residual matrix is ​​determined.

[0065] Based on the corrected class residual matrix, class probability estimation results, and preset correction parameters, combined with the correction operation, the corrected class probability estimation results for all samples are determined.

[0066] Based on the adjacency graph structure information of all samples, the corrected class probability estimation results, and the preset smoothing parameters, combined with the smoothing operation, the corrected and smoothed class probability estimation results of the corresponding samples are determined.

[0067] The category residuals obtained from the sample's class label and the corresponding class probability estimation result only contain the residual information of the sample itself. In this embodiment, the adjacency graph structure information, which represents the relationship between samples in the feature dimension, is fused with the category residuals to obtain category residuals that incorporate sample adjacency information. Then, the category residuals that incorporate sample adjacency information and the category residuals themselves are weighted according to preset residual parameters to obtain a corrected category residual matrix, which serves as the basis for error correction of the class probability estimation results.

[0068] Then, based on the corrected class residual matrix and the preset correction parameters, the correction amount of the class probability estimation results for all samples can be determined, and combined with the class probability estimation results themselves, the corrected class probability estimation results for all samples can be determined.

[0069] Then, the adjacency graph structure information of all samples and the corrected class probability estimation results are fused to obtain the corrected class probability estimation results that incorporate the sample adjacency information. The corrected class probability estimation results that incorporate the sample adjacency information are then weighted with the corrected class probability estimation results themselves according to the preset smoothing parameters to complete the smoothing iteration of the corrected class probability estimation results and obtain the corrected and smoothed class probability estimation results for the corresponding samples.

[0070] This embodiment fuses adjacency graph structure information with class residuals, combining the class residuals that incorporate sample adjacency information with the class residuals themselves to obtain a corrected class residual matrix. Then, based on the corrected class residual matrix and preset correction parameters, the correction amount of the class probability estimation result is determined, and combined with the class probability estimation result itself, the corrected class probability estimation result is determined. Subsequently, the adjacency graph structure information and the corrected class probability estimation result are fused, and the corrected class probability estimation result that incorporates sample adjacency information is combined with the corrected class probability estimation result itself. The corrected class probability estimation result is then smoothed and iterated to obtain the corrected and smoothed class probability estimation result for the corresponding sample, thereby improving the accuracy of the class probability estimation result.

[0071] Optionally, based on the adjacency graph structure information of all samples, the class residuals, and preset residual parameters, combined with a preset residual correction model, the corrected class residual matrix is ​​determined to include:

[0072] The preset residual parameters include a preset first residual parameter and a preset second residual parameter;

[0073] The adjacency graph structure information of all samples is standardized to obtain standardized adjacency graph structure information;

[0074] Based on the standardized adjacency graph structure information, category residuals, and preset first residual parameters, the corrected first category residual matrix is ​​determined;

[0075] Based on the category residuals and the preset second residual parameters, determine the corrected second category residual matrix;

[0076] Based on the corrected first-class residual matrix and the corrected second-class residual matrix, determine the corrected class residual matrix.

[0077] In this embodiment, an adjacency graph structure matrix can be determined based on the adjacency graph structure information of all samples. Then, the adjacency graph structure matrix of the samples can be standardized according to the degree matrix to obtain standardized adjacency graph structure information.

[0078] Based on the class residuals of all samples, the class residual matrix of the samples can be determined. Then, a preset first residual parameter is used as a weight parameter to fuse the standardized adjacency graph structure information and the class residual matrix, resulting in a corrected first class residual matrix that incorporates sample adjacency information. A preset second residual parameter is used as a weight parameter for the class residual matrix, resulting in a corrected second class residual matrix that represents the class residual information itself. Combining the corrected first and second class residual matrices yields the corrected class residual matrix. In this embodiment, the sum of the preset first and second residual parameters can be a preset value of 1.

[0079] For example, if we denote the adjacency graph structure information as A, then by standardizing the adjacency graph structure information A according to the degree matrix D, we can obtain the standardized adjacency graph structure information D. -1 / 2 AD -1 / 2 The class residual matrix E of the samples is determined based on the class residuals of all samples, and the preset first residual parameter is denoted as α. 11 Let the preset second residual parameter be denoted as α. 12 Then, the preset first residual parameter is used as the weight parameter to fuse the standardized adjacency graph structure information and the category residual matrix, resulting in the corrected first category residual matrix:

[0080]

[0081] In the formula, Let α be the corrected first-class residual matrix. 11 Here, A is the preset first residual parameter, D is the adjacency graph structure information, E is the degree matrix, and E is the class residual matrix.

[0082] Using the preset second residual parameter as the weight parameter of the class residual matrix, the corrected second class residual matrix is ​​obtained as follows:

[0083]

[0084] In the formula, For the corrected second-class residual matrix, α 12 is the preset second residual parameter, and E is the category residual matrix.

[0085] By combining the corrected first-class residual matrix and the corrected second-class residual matrix, the corrected class residual matrix can be determined as follows:

[0086]

[0087] In the formula, This is the corrected class residual matrix. This is the corrected first-class residual matrix. This is the corrected second-class residual matrix.

[0088] This embodiment standardizes the adjacency graph structure information, uses a preset first residual parameter as a weight parameter, and fuses the standardized adjacency graph structure information and the class residual matrix to obtain a corrected first class residual matrix that incorporates sample adjacency information. A preset second residual parameter is then used as a weight parameter for the class residual matrix to obtain a corrected second class residual matrix that represents the class residual information itself. Based on the corrected first and second class residual matrices, the corrected class residual matrix is ​​determined. By fusing sample adjacency information and sample information, the accuracy of the corrected class residual matrix is ​​improved.

[0089] Optionally, based on the corrected class residual matrix, class probability estimation results, and preset correction parameters, combined with the correction operation, the corrected class probability estimation results for all samples are determined as follows:

[0090] The product between the corrected class residual matrix and the preset correction parameters is calculated to obtain the correction amount of the class probability estimation results for all samples;

[0091] The sum of the corrections to the class probability estimates for all samples and the class probability estimates themselves is used to determine the corrected class probability estimates for all samples.

[0092] In order to correct the error of the category probability estimation results, this embodiment presets a correction parameter. The product between the corrected category residual matrix and the preset correction parameter is used as the correction amount of the category probability estimation results of all samples. Then, the correction amount of the category probability estimation results of all samples is combined with the category probability estimation results, and the sum of the two is determined as the corrected category probability estimation results of all samples.

[0093] For example, let the preset correction parameter be denoted as γ, and the estimated class probability of the i-th sample be denoted as p. i Where i = 1, 2, ..., M, and M is the number of samples. Then, based on the class probability estimation results of the M samples, the class probability estimation results P = [p1, p2, ..., p...] of all samples can be obtained. M The corrected class probability estimates for all samples are as follows:

[0094]

[0095] In the formula, P represents the corrected class probability estimates for all samples, and γ is the preset correction parameter. This is the corrected category residual matrix.

[0096] This embodiment obtains the correction amount of the class probability estimation results for all samples by multiplying the preset correction parameters and the corrected class residual matrix, and combines it with the class probability estimation results for all samples to obtain the corrected class probability estimation results for all samples. This completes the error correction of the class probability estimation results and improves the accuracy of the class probability estimation results.

[0097] Optionally, based on the adjacency graph structure information of all samples, the corrected class probability estimation results, and the preset smoothing parameters, combined with the smoothing operation, the corrected and smoothed class probability estimation results for the corresponding samples are determined, including:

[0098] The preset smoothing parameters include a first preset smoothing parameter and a second preset smoothing parameter;

[0099] The adjacency graph structure information of all samples is standardized to obtain standardized adjacency graph structure information;

[0100] Based on the standardized adjacency graph structure information, the corrected class probability estimation results, and the preset first smoothing parameter, the corrected and smoothed first class probability estimation results are determined.

[0101] Based on the corrected category probability estimation results and the preset second smoothing parameter, determine the corrected and smoothed second category probability estimation results;

[0102] Based on the corrected and smoothed first-class probability estimation results and the corrected and smoothed second-class probability estimation results, the corrected and smoothed class probability estimation results for the corresponding samples are determined.

[0103] In this embodiment, a preset first smoothing parameter is used as a weighting parameter to fuse the standardized adjacency graph structure information and the corrected class probability estimation results, resulting in a corrected and smoothed first class probability estimation result that incorporates sample adjacency information. A preset second smoothing parameter is used as a weighting parameter for the corrected class probability estimation result, resulting in a corrected and smoothed second class probability estimation result that represents the information inherent in the corrected class probability estimation result itself. Then, based on the corrected and smoothed first and second class probability estimation results, the corrected and smoothed class probability estimation results for all samples can be determined, thereby determining the corrected and smoothed class probability estimation result for each corresponding sample. In this embodiment, the sum of the preset first and second smoothing parameters can be a preset value of 1.

[0104] For example, let the preset first smoothing parameter be denoted as α. 21 Let the preset second smoothing parameter be denoted as α. 22Then, the preset first smoothing parameter is used as the weight parameter to fuse the standardized adjacency graph structure information and the corrected class probability estimation result, resulting in the corrected and smoothed first class probability estimation result:

[0105]

[0106] In the formula, This is the first-class probability estimate after correction and smoothing, α 21 Here, A is the preset first smoothing parameter, A is the adjacency graph structure information, and D is the degree matrix. This represents the corrected class probability estimates for all samples.

[0107] Using the preset second smoothing parameter as the weighting parameter of the corrected class probability estimation result, the corrected and smoothed second class probability estimation result is obtained as follows:

[0108]

[0109] In the formula, This is the corrected and smoothed second-class probability estimate, α. 22 The preset second smoothing parameter, This represents the corrected class probability estimates for all samples.

[0110] Based on the corrected and smoothed first-class probability estimation results and the corrected and smoothed second-class probability estimation results, the corrected and smoothed class probability estimation results are determined as follows:

[0111]

[0112] In the formula, To correct the smoothed class probability estimation results, This is the first-class probability estimate result after correction and smoothing. This is the second-class probability estimate result after correction and smoothing.

[0113] Then the corrected and smoothed class probability estimation results for all samples The corrected and smoothed class probability estimate for each sample can be determined.

[0114] This embodiment standardizes the adjacency graph structure information, uses a preset first smoothing parameter as a weighting parameter, and fuses the standardized adjacency graph structure information and the corrected class probability estimation results to obtain a first class probability estimation result that incorporates sample adjacency information. A preset second smoothing parameter is then used as a weighting parameter for the corrected class probability estimation result to obtain a second class probability estimation result that represents the information of the corrected class probability estimation result itself. Based on the first and second class probability estimation results, the corrected class probability estimation result for each sample can be determined. By fusing sample adjacency information and sample information, the accuracy of the corrected class probability estimation results is improved.

[0115] The above steps, which combine the adjacency graph structure information, class residuals, and class probability estimation results of all samples with a correction and smoothing operation, to obtain the corrected and smoothed class probability estimation results of the corresponding samples, improve the accuracy of the class probability estimation results by fusing the sample adjacency information and the class residuals of the samples themselves, correcting the class residuals, and then performing error correction and smoothing iterations on the class probability estimation results based on the adjacency graph structure information and the corrected class residuals.

[0116] Step S204: Based on the class probability prediction results from the previous iteration and the corrected and smoothed class probability estimation results from the current iteration, determine the class probability prediction results for the current iteration. The class probability prediction results from the current iteration are used to determine the model loss in the current iteration by combining the corrected and smoothed class probability estimation results and the class label. The model to be trained is trained according to the model loss to obtain the trained model.

[0117] Specifically, the class probability prediction result in this iteration integrates the class probability prediction result from the previous iteration and the corrected and smoothed class probability estimation result in this iteration. The class probability prediction result in the previous iteration integrates the class probability prediction result from the iteration before that and the corrected and smoothed class probability estimation result from the previous iteration, and so on. Therefore, the class probability prediction result integrates the corrected and smoothed class probability estimation results from this iteration and all iterations prior to this iteration. Thus, the class probability prediction result integrates the training results of the model to be trained in the historical iteration training process.

[0118] Meanwhile, the corrected and smoothed class probability estimation results fully consider the fusion of sample adjacency information and sample information. Therefore, the class probability prediction results in this iteration also incorporate the training results of the model to be trained in the historical iteration training process, as well as the mutual relationship between training samples in the feature dimension, which effectively improves the accuracy of the class probability prediction results.

[0119] Specifically, in this embodiment, the class probability prediction results from the previous iteration and the corrected and smoothed class probability estimation results from the current iteration are weighted and summed according to preset iteration parameters to determine the class probability prediction results for the current iteration. Then, the regularization loss is calculated based on the class probability prediction results and the corrected and smoothed class probability estimation results of all samples in the current iteration, and the class label vector is determined based on the class label of each sample. The cross-entropy loss is calculated based on the corrected and smoothed class probability estimation results and the class label vector of all samples. The model loss is determined based on the regularization loss and the cross-entropy loss to perform iterative training of the model to be trained until the model loss converges, and the trained model is obtained.

[0120] For example, let the preset first iteration parameter be denoted as β1, the preset second iteration parameter as β2, and the corrected and smoothed class probability estimate of the i-th sample in the k-th (k=2,3,...) iteration as β1. Let the predicted class probability of the i-th sample in the (k-1)-th iteration be denoted as Then, the predicted class probability of the i-th sample in the k-th iteration is:

[0121]

[0122] In the formula, This represents the predicted class probability of the i-th sample in the k-th iteration. This represents the corrected and smoothed class probability estimate for the i-th sample in the k-th iteration. β1 represents the predicted class probability of the i-th sample in the (k-1)-th iteration, where β1 is the preset first iteration parameter and β2 is the preset second iteration parameter.

[0123] Based on the predicted class probabilities of all samples in the k-th iteration and the corrected and smoothed class probability estimates, the regularization loss in the k-th iteration is calculated as follows:

[0124]

[0125] In the formula, L1(k) is the regularization loss in the k-th iteration. This represents the corrected and smoothed class probability estimate for the i-th sample in the k-th iteration. Let M be the predicted class probability of the i-th sample in the k-th iteration, and λ be the number of samples. k This is the preset loss parameter in the k-th iteration.

[0126] When the category label determines that the corresponding sample belongs to the s-th category (s = 1, 2, ..., S), the category label vector corresponding to the i-th sample can be denoted as Y. i= (0, ..., 1, ..., 0), where Y i The element at position s has a value of y. is =1, and the value of elements in other positions is 0, where S is the number of categories.

[0127] Based on the corrected and smoothed class probability estimates and class label vectors of all samples in the k-th iteration, the cross-entropy loss in the k-th iteration is calculated as follows:

[0128]

[0129] In the formula, L2(k) is the cross-entropy loss in the k-th iteration, and y is The category label vector Y corresponding to the i-th sample i The value of the element at the s-th position in the array. The corrected and smoothed class probability estimate of the i-th sample in the k-th iteration. The value of the element at position s, where M is the number of samples and S is the number of categories.

[0130] Based on the regularization loss and cross-entropy loss in the k-th iteration, the model loss in the k-th iteration can be determined as follows:

[0131] L(k) = L1(k) + L2(k)

[0132] In the formula, L(k) is the model loss in the k-th iteration, L1(k) is the regularization loss in the k-th iteration, and L2(k) is the cross-entropy loss in the k-th iteration.

[0133] This embodiment takes the classification of human poses in sample images as an example. Based on preset iteration parameters, the predicted class probability from the previous iteration and the corrected and smoothed estimated class probability from the current iteration are weighted and summed to determine the predicted class probability for the current iteration. Then, a regularization loss is calculated based on the predicted and corrected and smoothed class probability estimates of all human pose sample images in the current iteration. A class label vector is determined based on the class label of each human pose sample image. A cross-entropy loss is calculated based on the corrected and smoothed class probability estimates and class label vectors of all human pose sample images. The model loss is determined based on the regularization loss and cross-entropy loss to perform iterative training of the human pose classification model to be trained until the model loss converges, resulting in a trained human pose classification model.

[0134] Optionally, based on the class probability prediction results from the previous iteration and the corrected and smoothed class probability estimates from the current iteration, the class probability prediction results for the current iteration are determined as follows:

[0135] For the first iteration of training, obtain the preset initial class probability prediction results, and determine the class probability prediction results in the first iteration of training based on the preset initial class probability prediction vector, the preset iteration parameters, and the corrected and smoothed class probability estimation results in the first iteration of training.

[0136] For the k-th iteration training, k = 2, 3, ..., the class probability prediction result in the k-th iteration training is determined based on the class probability prediction result in the (k-1)-th iteration training, the preset iteration parameters, and the corrected and smoothed class probability estimation result in the k-th iteration training.

[0137] Since only the corrected and smoothed class probability estimation result of the first iteration is obtained after the first iteration of training on the sample, and the class probability prediction result of the previous iteration is not available, in order to facilitate the unified calculation of the class probability prediction result, this embodiment obtains the preset class probability prediction result and merges it with the corrected and smoothed class probability estimation result of the first iteration to obtain the class probability prediction result of the first iteration.

[0138] In one embodiment, in order not to affect the calculation result of the category probability prediction result, the preset category probability prediction result can be set as a zero vector.

[0139] Then, starting from the second iteration, the class probability prediction result in the k-th iteration can be determined by using the class probability prediction result in the (k-1)-th iteration training, the preset iteration parameters, and the corrected and smoothed class probability estimation result in the k-th iteration training.

[0140] This embodiment takes into account the differences between the first iteration and other iterations, and integrates the preset category probability prediction results with the corrected and smoothed category probability estimation results in the first iteration. It adaptively integrates and calculates the category probability prediction results for different iteration numbers, thereby improving the reliability and accuracy of the category probability prediction results.

[0141] The above steps, based on the class probability prediction results from the previous iteration and the corrected and smoothed class probability estimation results from the current iteration, determine the class probability prediction results for the current iteration. These predictions are then used to determine the model loss in the current iteration, combining the corrected and smoothed class probability estimation results with the class labels. The model is then trained based on this loss. This process ensures that the class probability prediction results in the current iteration incorporate the training results of the model in previous iterations, as well as the relationships between training samples in the feature dimension. This effectively improves the accuracy of the class probability prediction results. Furthermore, by determining the model loss based on the class probability prediction results, the corrected and smoothed class probability estimation results, and the class labels, the accuracy of the model is improved.

[0142] This embodiment obtains the class features and class probability estimation results of each sample in the current iteration when training the model using the training sample set. For any sample in the training sample set, the adjacency graph structure information of the sample is constructed based on the similarity of class features between the sample and other samples. Clean and unclean sample sets are selected from the training sample set. For any sample in the clean sample set, the class residual of the sample is determined based on the sample's class label and the corresponding class probability estimation result. The class residuals of all samples in the unclean sample set are set to preset values. Based on the adjacency graph structure information, class residuals, and class probability estimation results of all samples, combined with a correction and smoothing operation, the corrected and smoothed class probability estimation result of the corresponding sample is obtained. Based on the class probability prediction results in the previous iteration and the corrected and smoothed class probability in the current iteration... The estimation results determine the class probability prediction results in this iteration. These predictions are used to determine the model loss in this iteration, combining the corrected and smoothed class probability estimation results with the class labels. The model to be trained is then trained based on this loss, resulting in a trained model. By fusing sample adjacency information and the class residuals of the samples themselves, the class residuals are corrected. Furthermore, error correction and smoothing iterations are performed on the class probability estimation results based on the adjacency graph structure information and the corrected class residuals, improving the accuracy of the class probability estimation results. This allows the class probability prediction results in this iteration to simultaneously incorporate the training results of the model to be trained in previous iterations, as well as the relationships between training samples in the feature dimension, effectively improving the accuracy of the class probability prediction results and thus enhancing the accuracy of the model to be trained.

[0143] Embodiment 2 of the present invention provides an image classification method. This image classification method uses the model trained in Embodiment 1 of the present invention to classify the image to be classified, and may include the following steps:

[0144] The model acquires the image to be classified, inputs the image into the trained model, outputs the probability estimation result of the class of the image to be classified, and determines the image class based on the probability estimation result of the class of the image to be classified.

[0145] The image categories can be set according to the actual image classification task. For example, in scenarios such as product audience analysis and population aging analysis, the image classification task is to classify the facial attributes of the images to be classified. Image categories can be set to various facial attribute categories such as male, female, child, youth, middle-aged, and elderly, and the trained model is a trained facial attribute classification model. In scenarios such as gait analysis, video surveillance, and sports science, the image classification task is to classify the human posture of the images to be classified. Image categories can be set to various human posture categories such as stationary, walking, running, squatting, and jumping, and the trained model is a trained facial attribute classification model. The model is a pre-trained human posture classification model; in scenarios such as teaching evaluation, product sales, and personnel interviews, the image classification task is to classify the emotions of the people in the images to be classified. The image categories can be set to various human emotion categories such as happy, nervous, sad, disgusted, and bored. The trained model is a pre-trained human emotion classification model; in scenarios such as renting, decorating, and buying a house, the image classification task is to classify the room style of the images to be classified. The image categories can be set to various room styles such as pastoral, minimalist, classical, new Chinese, Mediterranean, and Southeast Asian. The trained model is a pre-trained room style classification model.

[0146] This embodiment takes the classification of human poses in an image to be classified as an example. After obtaining the human pose image to be classified, the human pose image is input into a trained human pose classification model for feature extraction and analysis. The model outputs a category probability estimation result for the human pose image. This category probability estimation result can represent the probability that the corresponding human pose image belongs to each human pose category. Then, the human pose category corresponding to the highest probability value in the category probability estimation result can be determined as the human pose category of the human pose image, thus completing the human pose classification task for the image to be classified.

[0147] It is understood that in the specific embodiments of this application, data related to facial images, human body images, room images, etc. are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0148] This embodiment uses the model trained in Embodiment 1 of the present invention to perform feature extraction and feature analysis on the image to be classified, outputs the category probability estimation result of the image to be classified, determines the image category of the image to be classified, and improves the classification accuracy of the image to be classified.

[0149] Corresponding to the sample category prediction method in the model training of the above embodiment, Figure 3 A structural block diagram of the sample category prediction device in model training provided in Embodiment 3 of the present invention is given. For ease of explanation, only the parts related to the embodiments of the present invention are shown.

[0150] See Figure 3 The model training includes a sample category prediction device comprising:

[0151] The graph structure information acquisition module 31 is used to acquire the category features and category probability estimation results of each sample in the current iteration when training the model to be trained using the training sample set. For any sample in the training sample set, the adjacency graph structure information of the sample is constructed based on the similarity of the category features between the sample and other samples.

[0152] The category residual determination module 32 is used to select a clean sample set and a non-clean sample set from the training sample set. For any sample in the clean sample set, the category residual of the sample is determined according to the sample's category label and the corresponding category probability estimation result. The category residual of all samples in the non-clean sample set is set to a preset value.

[0153] The correction and smoothing module 33 is used to obtain the corrected and smoothed class probability estimation result of the corresponding sample based on the adjacency graph structure information, class residuals, and class probability estimation results of all samples, combined with the correction and smoothing operation.

[0154] The iterative training module 34 is used to determine the class probability prediction result in the current iteration based on the class probability prediction result in the previous iteration and the corrected and smoothed class probability estimation result in the current iteration. The class probability prediction result in the current iteration is used to determine the model loss in the current iteration by combining the corrected and smoothed class probability estimation result and the class label. The model to be trained is trained according to the model loss to obtain the trained model.

[0155] Optionally, the graph structure information acquisition module 31 mentioned above includes:

[0156] The similarity calculation submodule is used to calculate the similarity of class features between any sample and other samples in the training sample set.

[0157] The sample selection submodule is used to select a preset number of samples from other samples based on similarity and determine them as similar samples. Samples that are not selected from other samples are determined as dissimilar samples.

[0158] The graph structure information construction submodule is used to construct the adjacency graph structure information of samples based on similar samples, a first preset relationship value between samples and similar samples, and a second preset relationship value between dissimilar samples and dissimilar samples.

[0159] Optionally, the above-mentioned correction smoothing module 33 includes:

[0160] The first correction submodule is used to determine the corrected class residual matrix based on the adjacency graph structure information of all samples, the class residuals, and the preset residual parameters, combined with the preset residual correction model.

[0161] The second correction submodule is used to determine the corrected class probability estimation results for all samples based on the corrected class residual matrix, class probability estimation results, and preset correction parameters, combined with the correction operation.

[0162] The smoothing submodule is used to determine the corrected and smoothed class probability estimate of the corresponding sample based on the adjacency graph structure information of all samples, the corrected class probability estimate, and the preset smoothing parameters, combined with the smoothing operation.

[0163] Optionally, the aforementioned first correction submodule includes:

[0164] The residual parameter determination unit is used to determine preset residual parameters, including preset first residual parameters and preset second residual parameters;

[0165] The graph structure information standardization unit is used to standardize the adjacency graph structure information of all samples to obtain standardized adjacency graph structure information.

[0166] The first category residual correction unit is used to determine the corrected first category residual matrix based on the standardized adjacency graph structure information, category residuals and preset first residual parameters.

[0167] The second category residual correction unit is used to determine the corrected second category residual matrix based on the category residuals and the preset second residual parameters.

[0168] The category residual correction unit is used to determine the corrected category residual matrix based on the corrected first category residual matrix and the corrected second category residual matrix.

[0169] Optionally, the second correction submodule mentioned above includes:

[0170] The correction calculation unit is used to calculate the product between the corrected class residual matrix and the preset correction parameters to obtain the correction amount of the class probability estimation results for all samples.

[0171] The category probability estimation result correction unit is used to determine the corrected category probability estimation result for all samples by summing the correction amount of the category probability estimation result with the category probability estimation result.

[0172] Optionally, the above smoothing submodule includes:

[0173] The smoothing parameter determination unit is used to determine preset smoothing parameters, including preset first smoothing parameters and preset second smoothing parameters;

[0174] The graph structure information standardization unit is used to standardize the adjacency graph structure information of all samples to obtain standardized adjacency graph structure information.

[0175] The first smoothing unit is used to determine the corrected and smoothed first class probability estimation result based on the standardized adjacency graph structure information, the corrected class probability estimation result and the preset first smoothing parameter.

[0176] The second smoothing unit is used to determine the corrected and smoothed second category probability estimation result based on the corrected category probability estimation result and the preset second smoothing parameter;

[0177] The category probability estimation result smoothing unit is used to determine the corrected smoothed category probability estimation result for the corresponding sample based on the corrected smoothed first category probability estimation result and the corrected smoothed second category probability estimation result.

[0178] Optionally, the iterative training module 34 mentioned above includes:

[0179] The first iteration submodule is used to obtain the preset initial class probability prediction result for the first iteration of training, and determine the class probability prediction result in the first iteration of training based on the preset initial class probability prediction vector, the preset iteration parameters and the corrected and smoothed class probability estimation result in the first iteration of training.

[0180] The second iteration submodule is used to determine the class probability prediction result in the k-th iteration training for k=2,3,... based on the class probability prediction result in the (k-1)-th iteration training, the preset iteration parameters, and the corrected and smoothed class probability estimation result in the k-th iteration training.

[0181] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4 Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor, which, when executed by the processor, implements the steps in the embodiments of the method for predicting sample categories during the training of any of the above models.

[0182] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0183] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0184] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0185] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0186] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0187] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0188] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0189] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0190] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0191] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for predicting a sample class in model training, characterized in that, The methods for predicting sample categories during model training include: When training the model to be trained using the training sample set, the category features and category probability estimation results of each sample in this iteration are obtained. For any sample in the training sample set, the adjacency graph structure information of the sample is constructed based on the similarity of the category features between the sample and other samples. Here, the model to be trained is a human pose classification model to be trained, the training sample set is a noisy human pose image dataset, the category features are the feature information extracted for each human pose sample image, the category probability estimation result represents the probability that the corresponding human pose sample image belongs to each human pose category, and the adjacency graph structure information is constructed based on the similarity of the category features between the human pose sample image and other human pose sample images. A clean sample set and a non-clean sample set are selected from the training sample set. For any sample in the clean sample set, the class residual of the sample is determined according to the class label of the sample and the corresponding class probability estimation result. The class residuals of all samples in the non-clean sample set are set to a preset value. Based on the adjacency graph structure information of all human pose sample images and the category residual, a correction operation is used to correct the error in the category probability estimation result. Based on the adjacency graph structure information of all human pose sample images, a smoothing operation is used to smooth the category probability estimation result, thereby obtaining the corrected and smoothed category probability estimation result of the corresponding human pose sample image. Based on the class probability prediction results from the previous iteration and the corrected and smoothed class probability estimation results from the current iteration, the class probability prediction results for the current iteration are determined. These predicted results are used to determine the model loss in the current iteration by combining the corrected and smoothed class probability estimation results with the class labels. The model to be trained is then trained based on the model loss to obtain a trained model.

2. The method for predicting sample categories during model training according to claim 1, characterized in that, The step of constructing the adjacency graph structure information of any sample in the training sample set based on the similarity of the category features between the sample and other samples includes: For any sample in the training sample set, calculate the similarity of the category features between the sample and other samples; Based on the similarity, a preset number of samples are selected from the other samples and determined as similar samples of the sample; samples that are not selected from the other samples are determined as dissimilar samples of the sample. Based on the similar samples, the first preset relationship value between the sample and the similar samples, and the second preset relationship value between the dissimilar samples and the dissimilar samples, the adjacency graph structure information of the sample is constructed.

3. The method for predicting sample categories during model training according to claim 1, characterized in that, The step involves correcting errors in the class probability estimation result by combining the adjacency graph structure information of all human pose sample images and the class residuals with a correction operation, and smoothing the class probability estimation result by combining the adjacency graph structure information of all human pose sample images with a smoothing operation, to obtain the corrected and smoothed class probability estimation result for the corresponding human pose sample image, including: Based on the adjacency graph structure information of all samples, the category residuals, and the preset residual parameters, combined with the preset residual correction model, the corrected category residual matrix is ​​determined. Based on the corrected category residual matrix, the category probability estimation results, and the preset correction parameters, combined with the correction operation, the corrected category probability estimation results for all samples are determined. Based on the adjacency graph structure information of all samples, the corrected class probability estimation results, and the preset smoothing parameters, combined with the smoothing operation, the corrected and smoothed class probability estimation results of the corresponding samples are determined.

4. The method for predicting sample categories during model training according to claim 3, characterized in that, The step of determining the corrected class residual matrix based on the adjacency graph structure information of all samples, the class residuals, and preset residual parameters, combined with a preset residual correction model, includes: The preset residual parameters include a preset first residual parameter and a preset second residual parameter; The adjacency graph structure information of all samples is standardized to obtain standardized adjacency graph structure information; Based on the standardized adjacency graph structure information, the category residuals, and the preset first residual parameters, the corrected first category residual matrix is ​​determined; Based on the category residuals and the preset second residual parameters, determine the corrected second category residual matrix; The corrected category residual matrix is ​​determined based on the corrected first category residual matrix and the corrected second category residual matrix.

5. The method for predicting sample categories during model training according to claim 3, characterized in that, The step of determining the corrected class probability estimation results for all samples based on the corrected class residual matrix, the class probability estimation results, and preset correction parameters, combined with the correction operation, includes: Calculate the product between the corrected class residual matrix and the preset correction parameter to obtain the correction amount of the class probability estimation results for all samples; The sum of the corrections to the class probability estimates for all samples and the class probability estimates themselves is used to determine the corrected class probability estimates for all samples.

6. The method for predicting sample categories during model training according to claim 3, characterized in that, The step of determining the corrected and smoothed class probability estimation result for the corresponding sample based on the adjacency graph structure information of all samples, the corrected class probability estimation result, and the preset smoothing parameters, combined with a smoothing operation, includes: The preset smoothing parameters include a preset first smoothing parameter and a preset second smoothing parameter; The adjacency graph structure information of all samples is standardized to obtain standardized adjacency graph structure information; Based on the standardized adjacency graph structure information, the corrected category probability estimation result, and the preset first smoothing parameter, the corrected and smoothed first category probability estimation result is determined. Based on the corrected category probability estimation result and the preset second smoothing parameter, determine the corrected and smoothed second category probability estimation result; Based on the first category probability estimation result and the second category probability estimation result after correction and smoothing, the category probability estimation result of the corresponding sample is determined.

7. The method for predicting sample categories during model training according to claim 1, characterized in that, The step of determining the class probability prediction result for the current iteration based on the class probability prediction result from the previous iteration and the corrected and smoothed class probability estimation result for the current iteration includes: For the first iteration of training, a preset initial class probability prediction result is obtained. Based on the preset initial class probability prediction vector, the preset iteration parameters, and the corrected and smoothed class probability estimation result in the first iteration of training, the class probability prediction result in the first iteration of training is determined. For the k-th iteration training, k=2,3,..., the class probability prediction result in the k-th iteration training is determined based on the class probability prediction result in the (k-1)-th iteration training, the preset iteration parameters, and the corrected and smoothed class probability estimation result in the k-th iteration training.

8. A device for predicting sample categories during model training, characterized in that, The device for predicting sample categories during model training includes: The graph structure information acquisition module is used to acquire the category features and category probability estimation results of each sample in the current iteration when training the model to be trained using the training sample set. For any sample in the training sample set, the adjacency graph structure information of the sample is constructed based on the similarity of the category features between the sample and other samples. Here, the model to be trained is a human pose classification model to be trained, the training sample set is a noisy human pose image dataset, the category features are the feature information extracted for each human pose sample image, the category probability estimation result represents the probability that the corresponding human pose sample image belongs to each human pose category, and the adjacency graph structure information is constructed based on the similarity of the category features between the human pose sample image and other human pose sample images. The category residual determination module is used to select a clean sample set and a non-clean sample set from the training sample set, and for any sample in the clean sample set, determine the category residual of the sample based on the category label of the sample and the corresponding category probability estimation result, and set the category residual of all samples in the non-clean sample set to a preset value. The correction and smoothing module is used to correct the error in the category probability estimation result by combining the adjacency graph structure information of all human posture sample images and the category residual with the correction operation, and to smooth the category probability estimation result by combining the adjacency graph structure information of all human posture sample images with the smoothing operation, so as to obtain the corrected and smoothed category probability estimation result of the corresponding human posture sample image. The iterative training module is used to determine the class probability prediction result in the current iteration based on the class probability prediction result in the previous iteration and the corrected and smoothed class probability estimation result in the current iteration. The class probability prediction result in the current iteration is used to determine the model loss in the current iteration by combining the corrected and smoothed class probability estimation result and the class label. The model to be trained is trained according to the model loss to obtain the trained model.

9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for predicting sample categories during model training as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for predicting sample categories during model training as described in any one of claims 1 to 7.

Images