Image classification model training data selection method and device and image classification method

By using the mutual information index as the evaluation metric for the Shapley formula, the problem of low efficiency in selecting training data for image classification models in existing technologies is solved, achieving efficient and low-cost selection of training data and improving the classification accuracy of the model.

CN117197618BActive Publication Date: 2026-03-27QINGRUN DIGITAL (XIAN) DATA ASSET APPRAISAL CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-26
Publication Date
2026-03-27

AI Technical Summary

Technical Problem

In the training of image classification models, existing technologies, such as permutation sampling algorithms, result in low computational efficiency, high implementation costs, and difficulty in fairly evaluating the data value of image datasets.

Method used

Mutual information index is used as the evaluation index of Shapley formula. The mutual information index is obtained by calculating the data feature information entropy, data label information entropy and feature-label joint value information entropy of image dataset, and the training data is selected by Shapley formula.

Benefits of technology

It reduces the time and cost of selecting training data, improves the training efficiency and accuracy of image classification models, and reduces the dependence on model training.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117197618B_ABST
    Figure CN117197618B_ABST
Patent Text Reader

Abstract

The application belongs to the field of image processing, and discloses an image classification model training data selection method and device and an image classification method, which comprises the following steps: obtaining data feature information entropy, data label information entropy and feature and label joint value information entropy of each subset of all candidate training data sets according to data features and data labels of each training data in each candidate training data set, and calculating mutual information indexes of each subset of all candidate training data sets; taking the mutual information index as an evaluation index of a Shapley formula, and obtaining Shapley values of each candidate training data set through the Shapley formula; and selecting the first preset candidate training data sets in the order from large to small of the Shapley values of each candidate training data set as image classification model training data. The mutual information index between data features and labels is calculated as the evaluation index of the Shapley formula, the model does not need to be repeatedly trained, the time for calculating the mutual information index is very short, and the cost is greatly reduced.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, and relates to an image classification model training data selection method and device and an image classification method. BACKGROUND

[0002] Emerging big data in various industries has become a driving force for technological and economic development. The role of data is to train various models, and through the models, more industries are empowered. For an image classification model, multiple image data sets are usually faced during training. Based on training conditions and time requirements, all image data sets cannot be used for training of the image classification model in actual situations, and usually a part of the image data sets is selected for training. The quality of the image data set determines the quality of the model. In order to select image data sets with better quality, a method for evaluating the data value of the image data set is needed. The data value of the image data set depends largely on its contribution to a specific machine learning model. The core challenge here is how to fairly and accurately evaluate the contribution of each image data in the image data set to the learning algorithm of a specific performance indicator.

[0003] The most widely used algorithm to solve this problem is the permutation sampling algorithm (also known as Monte Carlo sampling), which first randomly permutes the image data sets, then scans each element from the first element to the last element in the permutation, calculates the marginal contribution of each element to the set of elements before it, and finally repeats the same process on multiple permutations and takes the average of all marginal contributions as an approximation of the Shapley value. The Shapley value is then used to evaluate the data value, and the image data set is selected. However, the marginal contribution of this method needs to be based on the training of the image classification model, and the image classification model needs to be trained repeatedly. When the number of image data sets is large, the number of training times increases exponentially. Assuming the number of image data sets is N, the image classification model needs to be trained N factorial times, the calculation efficiency is very low, and the training time of a single image data set increases when the number of image data in a single image data set is large, resulting in high implementation cost. SUMMARY

[0004] The present application aims to overcome the shortcomings of the prior art and provide an image classification model training data selection method, device and image classification method.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] In a first aspect of the present application, an image classification model training data selection method is provided, comprising:

[0007] obtaining each candidate training data set of the image classification model, and obtaining the data features and data labels of each training data in each candidate training data set;

[0008] According to the data features and data labels of the training data in each of the candidate training data sets, data feature information entropy, data label information entropy, and feature and label joint value information entropy of each subset of all the candidate training data sets are obtained;

[0009] According to the data feature information entropy, data label information entropy, and feature and label joint value information entropy of each subset of all the candidate training data sets, mutual information indicators of each subset of all the candidate training data sets are obtained;

[0010] The mutual information indicators are taken as evaluation indicators of the Shapley formula, and Shapley values of each of the candidate training data sets are obtained through the Shapley formula;

[0011] According to an order from large to small of the Shapley values of each of the candidate training data sets, a preset number of candidate training data sets are selected as image classification model training data.

[0012] Optionally, the obtaining of the data feature information entropy, the data label information entropy, and the feature and label joint value information entropy of each subset of all the candidate training data sets according to the data features and data labels of the training data in each of the candidate training data sets comprises:

[0013] Each subset of all the candidate training data sets is traversed, and data feature information entropy H(dest) of a current subset is obtained through the following formula:

[0014]

[0015] wherein, destp i is a probability of an i-th data feature in the data features of each training data in the current subset, and t is a total number of the data features of each training data in the current subset;

[0016] Each subset of all the candidate training data sets is traversed, and data label information entropy H(src) of a current subset is obtained through the following formula:

[0017]

[0018] wherein, srcp i is a probability of an i-th data label in the data labels of each training data in the current subset, and m is a total number of the data labels of each training data in the current subset;

[0019] Each subset of all the candidate training data sets is traversed, and feature and label joint value information entropy H(dest-src) of a current subset is obtained through the following formula:

[0020]

[0021] wherein, dest-srcpi Pi is the probability of the i-th feature and label joint value of the feature and label joint value of each training data in the current subset, k is the total number of feature and label joint values of each training data in the current subset, and the feature and label joint value is obtained by appending the data label to the data feature.

[0022] Optionally, before the data feature is discretized, the data feature information entropy, the data label information entropy, and the feature and label joint value information entropy of each subset of all candidate training data sets are obtained according to the data features and data labels of each training data in each candidate training data set.

[0023] Optionally, the discretization of the data feature includes discretizing the data feature by a quantity-based binning method, an equidistant binning method, or a clustering method.

[0024] Optionally, the mutual information index of each subset of all candidate training data sets is obtained according to the data feature information entropy, the data label information entropy, and the feature and label joint value information entropy of each subset of all candidate training data sets.

[0025] The mutual information index of the current subset is obtained by the following formula by traversing each subset of all candidate training data sets.

[0026] I(Q) = H(dest) + H(src) - H(dest-src)

[0027] where I(Q) is the mutual information index of the current subset Q, H(dest) is the data feature information entropy of the current subset, H(src) is the data label information entropy of the current subset, and H(dest-src) is the feature and label joint value information entropy of the current subset.

[0028] Optionally, the mutual information index is used as the evaluation index of the Shapley formula, and the Shapley value of each candidate training data set is obtained by the Shapley formula.

[0029] The Shapley value of each candidate training data set is obtained by the following formula.

[0030]

[0031] where φ i (I) is the Shapley value of the i-th candidate training data set, n is the total number of candidate training data sets, S is a subset of all candidate training data sets, |S| is the number of candidate training data sets contained in S, d i is the i-th candidate training data set, D n is the set of all candidate training data sets, I(S∪{d i}) is the Shapley value of S∪{d iThe mutual information index of S is I(S).

[0032] In a second aspect, the present application provides an image classification method, comprising:

[0033] An image to be classified is obtained, and the image to be classified is input into a preset image classification model to obtain an image classification result of the image to be classified.

[0034] The image classification model is trained based on the image classification model training data selected by the image classification model training data selection method.

[0035] In a third aspect, the present application provides an image classification model training data selection system, comprising:

[0036] A data acquisition module is configured to acquire each training data set to be selected of an image classification model, and acquire data features and data labels of each training data in each training data set to be selected.

[0037] An information entropy calculation module is configured to acquire data feature information entropy, data label information entropy and feature and label joint value information entropy of each subset of all training data sets to be selected according to the data features and data labels of each training data in each training data set to be selected.

[0038] A mutual information calculation module is configured to acquire a mutual information index of each subset of all training data sets to be selected according to the data feature information entropy, data label information entropy and feature and label joint value information entropy of each subset of all training data sets to be selected.

[0039] A Shapley value calculation module is configured to take the mutual information index as an evaluation index of a Shapley formula, and acquire a Shapley value of each training data set to be selected by the Shapley formula.

[0040] A selection module is configured to select the first preset training data sets to be selected as image classification model training data according to the order from large to small of the Shapley values of each training data set to be selected.

[0041] In a fourth aspect, the present application provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image classification model training data selection method when executing the computer program.

[0042] In a fifth aspect, the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the image classification model training data selection method.

[0043] Compared with the prior art, the present application has the following beneficial effects:

[0044] The image classification model training data selection method of the present application, by obtaining the data characteristics and data labels of each training data in each candidate training data set, then according to the data characteristics and data labels of each training data in each candidate training data set, obtaining the data feature information entropy, data label information entropy and feature and label joint value information entropy of each subset of all candidate training data sets, and then calculating the mutual information index of each subset of all candidate training data sets, and taking the mutual information index as the evaluation index of the shapley formula, obtaining the shapley value of each candidate training data set through the shapley formula, and finally selecting the first preset candidate training data set as the image classification model training data according to the order of the shapley value of each candidate training data set from large to small. The data information entropy is proportional to the size of the data set, the information entropy is proportional to the richness of the data, the data information entropy of the data set is a monotonic increasing function of the classification accuracy of the classification model trained based on the data set, and the calculation of the information entropy is efficient. These characteristics make it very suitable to replace the evaluation index based on model training with the mutual information evaluation index based on information entropy. By calculating the mutual information index between the data characteristics and the labels as the evaluation index of the shapley formula, the model does not need to be repeatedly trained, the time for calculating the mutual information index is very fast, and even if the data volume is large, the cost can be greatly reduced. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 The image classification model training data selection method flow chart of the present application embodiment.

[0046] Figure 2 The image classification model training data selection system structure block diagram of the present application embodiment. DETAILED DESCRIPTION

[0047] In order to make the person skilled in the art better understand the present application scheme, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor shall belong to the scope of protection of the present application.

[0048] It is to be understood that the terminology "first", "second", and the like used throughout this specification and the accompanying drawings is merely for distinguishing between similar objects of the same type, and is not necessarily intended to convey a specific order or sequence among the described steps. It is to be understood that the use of the terms first, second, etc., was merely intended to differentiate between two separately discussed objects and not necessarily for describing a specific sequential order. It is to be understood that the use of the terms first, second, etc., was merely intended to differentiate between two separately discussed objects and not necessarily for describing a specific sequential order. Furthermore, the terms "comprise", "comprising", "include", "including", and the like used throughout this specification are used in the sense of "including" and not the sense of "consisting of", for example, a process, method, system, product, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements, but can include other steps or elements not expressly listed or inherent to such process, method, system, product, or apparatus.

[0049] The application will be further described in details below with reference to the accompanying drawings:

[0050] Referring to Figure 1 In an embodiment of the present application, a method for selecting training data of an image classification model is provided. The method uses mutual information as an evaluation index of the Shapley formula to effectively reduce the difficulty and time of selecting training data. Specifically, the method comprises the following steps:

[0051] S1: Obtain each candidate training data set of the image classification model, and obtain the data features and data labels of each training data in each candidate training data set.

[0052] S2: Obtain the data feature information entropy, data label information entropy, and feature and label joint value information entropy of each subset of all candidate training data sets according to the data features and data labels of each training data in each candidate training data set.

[0053] S3: Obtain the mutual information index of each subset of all candidate training data sets according to the data feature information entropy, data label information entropy, and feature and label joint value information entropy of each subset of all candidate training data sets.

[0054] S4: Use the mutual information index as an evaluation index of the Shapley formula to obtain the Shapley value of each candidate training data set by the Shapley formula.

[0055] S5: Select the first pre-set candidate training data set as the training data of the image classification model according to the order of the Shapley values of each candidate training data set from large to small.

[0056] The image classification model training data selection method of the application, by acquiring the data characteristics and data labels of each training data in each candidate training data set, then according to the data characteristics and data labels of each training data in each candidate training data set, acquiring the data characteristic information entropy, data label information entropy and feature and label joint value information entropy of each subset of all candidate training data sets, and then calculating the mutual information index of each subset of all candidate training data sets, and taking the mutual information index as the evaluation index of the Shapley formula, acquiring the Shapley value of each candidate training data set through the Shapley formula, and finally selecting the first preset candidate training data set as the image classification model training data according to the order from large to small of the Shapley value of each candidate training data set. The data information entropy is proportional to the size of the data set, the information entropy is proportional to the richness of the data, the data information entropy of the data set is a monotonic increasing function of the classification accuracy of the classification model trained based on the data set, and the calculation of the information entropy is efficient. These characteristics make it very suitable to replace the evaluation index based on model training with the mutual information evaluation index based on information entropy. By calculating the mutual information index between the data characteristics and the labels as the evaluation index of the Shapley formula, the model does not need to be repeatedly trained, the time for calculating the mutual information index is very fast, and even if the data volume is large, the cost can be greatly reduced.

[0057] Shapley value: Shapley, one of the founders of game theory, has high attainments in the study of non-strategic multi-person cooperation benefit distribution. The Shapley value method created by him is of great help to solve the problem of cooperative benefit distribution, and is a reasonable and scientific distribution method. Problems such as cost allocation and profit and loss allocation that are difficult to solve can be solved through the Shapley value method. Its goal is to construct a compromise utility distribution scheme that considers the requirements of all parties in conflict, thereby ensuring the fairness of the distribution.

[0058] Information entropy: It is a basic concept of information theory. It describes the uncertainty of the occurrence of each possible event of the information source. In the 1940s, Shannon borrowed the concept of thermodynamics and called the average amount of information after excluding redundancy in information as "information entropy", and gave the mathematical expression for calculating information entropy. The proposal of information entropy solves the problem of quantitative measurement of information.

[0059] Data discretization: When the data is continuous real number, the data value can have infinite possibilities. In order to facilitate operation, the data value is approximated according to certain rules, so that the data value becomes a finite value.

[0060] Data binning: a way of data discretization, data is put into different boxes according to the value size, and then the data value in each box is regarded as the same value.

[0061] Information entropy is a concept in information theory for measuring the amount of information. For a field or all fields in a table, information entropy calculation can obtain the minimum information flux of the table. If the source symbol has n values: U1,..., Ui,..., Un, the corresponding probability is: P1,..., Pi,..., Pn, and the occurrence of each symbol is independent of each other. At this time, the average uncertainty of the source should be the statistical average (E) of the uncertainty of a single symbol -logP i ,..., P n , and the occurrence of each symbol is independent of each other. At this time, the average uncertainty of the source should be the statistical average (E) of the uncertainty of a single symbol -logP i , which can be called information entropy, that is:

[0062] In a possible implementation, the data feature information entropy, data label information entropy, and feature and label joint value information entropy of each subset of all candidate training data sets are obtained according to the data features and data labels of each training data in each candidate training data set, which includes:

[0063] The data feature information entropy H(dest) of the current subset is obtained by traversing each subset of all candidate training data sets through the following formula:

[0064]

[0065] Wherein, destp i is the probability of the i-th data feature in the data features of each training data in the current subset, and t is the total number of data features of each training data in the current subset.

[0066] The data label information entropy H(src) of the current subset is obtained by traversing each subset of all candidate training data sets through the following formula:

[0067]

[0068] Wherein, srcp i is the probability of the i-th data label in the data labels of each training data in the current subset, and m is the total number of data labels of each training data in the current subset.

[0069] The feature and label joint value information entropy H(dest-src) of the current subset is obtained by traversing each subset of all candidate training data sets through the following formula:

[0070]

[0071] Wherein, dest-srcp i is the probability of the i-th feature and label joint value in the feature and label joint values of each training data in the current subset, and k is the total number of feature and label joint values of each training data in the current subset, and the feature and label joint value is obtained by appending the data label to the data feature.

[0072] Specifically, the data features of the training data in each of the candidate training data sets of the image classification model can be pixel values of the pixels of the images, and the data labels can be label values of the images.

[0073] Referring to Table 1, the data features and the data labels of a data set are schematically shown.

[0074] Table 1 Data features and data labels of a data set

[0075] ID (Data Number) Dest (Data Feature) Src (Data Tag) 1 01 0 2 00 1 3 11 1 4 10 0

[0076] Referring to Table 2, the feature and label joint values of the data set in Table 1 are shown.

[0077] Table 2 Feature and label joint values of a data set

[0078] Dest-Src (Feature and Tag Joint Value) 010 001 111 100

[0079] In a possible implementation, before the data feature information entropy, the data label information entropy and the feature and label joint value information entropy of each subset of all the candidate training data sets are obtained according to the data features and the data labels of the training data in each of the candidate training data sets, the data features are discretized.

[0080] Optionally, the data features can be discretized by a quantity-based binning method, an equidistant binning method or a clustering method.

[0081] Specifically, the data features need to be discretized when the information entropy is calculated. In this embodiment, the equidistant binning method is used to discretize the data. For example, a batch of data features is [0.023, 0.153, 0.112, 0.054, 0.082, 0.175, 0.012, 0.126, 0.009, 0.134], the maximum value of the data features is 0.2, and the minimum value is 0. Then, according to the equidistant binning method, the data features are discretized into four bins, and the values in each bin are replaced by the serial number of the bin. The discretized data features become [1, 4, 3, 2, 2, 4, 1, 3, 1, 3].

[0082] In a possible implementation, the mutual information index of each subset of all the candidate training data sets is obtained according to the data feature information entropy, the data label information entropy and the feature and label joint value information entropy of each subset of all the candidate training data sets, including: traversing each subset of all the candidate training data sets, and obtaining the mutual information index of the current subset by the following formula:

[0083] I(Q) = H(dest) + H(src) - H(dest-src)

[0084] Wherein, I(Q) is the mutual information index of the current subset Q, H(dest) is the data feature information entropy of the current subset, H(src) is the data label information entropy of the current subset, and H(dest-src) is the feature and label joint value information entropy of the current subset.

[0085] In a possible implementation, the mutual information index is taken as the evaluation index of the Shapley formula, and the Shapley value of each candidate training data set is obtained by the Shapley formula.

[0086] The Shapley value of each candidate training data set is obtained by the following formula:

[0087]

[0088] Wherein, φ i (I) is the Shapley value of the i-th candidate training data set, n is the total number of candidate training data sets, S is a subset of all candidate training data sets, |S| is the number of candidate training data sets contained in S, d i is the i-th candidate training data set, D n is the set of all candidate training data sets, I(S∪{d i}) is the mutual information index of S∪{d i}.

[0089] The present application is different from the model-based calculation method, and does not need to repeatedly train the model, but only needs to calculate the mutual information between the data features and the labels, and the calculation time of the mutual information is very fast, even if the data amount is very large, so that the cost can be greatly reduced, and the high-quality data selection has greater utilization value, and the quality of the data determines the quality of the model, and the data with better quality can be selected.

[0090] The data information entropy is proportional to the size of the data set, the information entropy is proportional to the richness of the data, the data information entropy of the data set is a monotonic increasing function of the classification accuracy of the classifier trained based on the data set, and the calculation of the information entropy is efficient, so that the mutual information evaluation index based on the information entropy is replaced by the evaluation index based on the model training, which is very suitable.

[0091] In another embodiment of the present application, an image classification method is provided, which specifically comprises the following steps:

[0092] An image to be classified is obtained, and the image to be classified is input into a preset image classification model to obtain an image classification result of the image to be classified, wherein the image classification model is trained based on the image classification model training data selected by the image classification model training data selection method.

[0093] The following is an embodiment of the device of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the device embodiment, please refer to the method embodiment of the present application.

[0094] Referring to Figure 2 In another embodiment of the present application, an image classification model training data selection system is provided, which can be used to implement the image classification model training data selection method described above. Specifically, the image classification model training data selection system includes a data acquisition module, an information entropy calculation module, a mutual information calculation module, a Shapley value calculation module, and a selection module.

[0095] The data acquisition module is configured to acquire each candidate training data set of the image classification model, and acquire data features and data labels of each training data in each candidate training data set. The information entropy calculation module is configured to acquire data feature information entropy, data label information entropy, and feature and label joint value information entropy of each subset of all candidate training data sets according to the data features and data labels of each training data in each candidate training data set. The mutual information calculation module is configured to acquire mutual information indicators of each subset of all candidate training data sets according to the data feature information entropy, data label information entropy, and feature and label joint value information entropy of each subset of all candidate training data sets. The Shapley value calculation module is configured to take the mutual information indicators as evaluation indicators of the Shapley formula, and acquire Shapley values of each candidate training data set through the Shapley formula. The selection module is configured to select the first pre-set candidate training data set as the image classification model training data according to the order of the Shapley values of each candidate training data set from large to small.

[0096] All related contents of each step involved in the foregoing embodiment of the image classification model training data selection method can be cited to the function description of the function module corresponding to the image classification model training data selection device in the embodiment of the present application, which will not be repeated here.

[0097] The division of the modules in the embodiment of the present application is illustrative, and is only a logical function division. In actual implementation, another division mode can be used. In addition, each function module in each embodiment of the present application can be integrated in one processor, or can be physically separated, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module.

[0098] In another embodiment of the present application, a computer device is provided, which comprises a processor and a memory, the memory is configured to store a computer program, the computer program comprises program instructions, and the processor is configured to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement a corresponding method process or a corresponding function; the processor in the embodiments of the present application can be used for the operation of the image classification model training data selection method.

[0099] In another embodiment of the present application, the present application further provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in a computer device, and is configured to store programs and data. It can be understood that the computer readable storage medium herein can include an internal storage medium in the computer device, and of course can also include an expansion storage medium supported by the computer device. The computer readable storage medium provides a storage space, and the storage space stores an operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. One or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the image classification model training data selection method in the above embodiments.

[0100] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0101] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0102] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0104] Finally, it should be noted that the above-mentioned embodiments are merely intended for describing the technical solutions of the present application, but not for limiting it. Although the present application is described in detail with reference to the above embodiments, those skilled in the field should understand that the specific embodiments of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and any modification or equivalent replacement without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A method for selecting training data of an image classification model, characterized in that, The method comprises the following steps: obtaining each candidate training data set of an image classification model, and obtaining data features and data labels of each training data in each candidate training data set; obtaining data feature information entropy, data label information entropy and feature and label joint value information entropy of each subset of all candidate training data sets according to the data features and data labels of each training data in each candidate training data set; obtaining mutual information indicators of each subset of all candidate training data sets according to the data feature information entropy, data label information entropy and feature and label joint value information entropy of each subset of all candidate training data sets; taking the mutual information indicators as evaluation indicators of a Shapley formula, and obtaining Shapley values of each candidate training data set through the Shapley formula; selecting the first preset candidate training data sets in the order from large to small of the Shapley values of each candidate training data set as image classification model training data; the step of obtaining mutual information indicators of each subset of all candidate training data sets according to the data feature information entropy, data label information entropy and feature and label joint value information entropy of each subset of all candidate training data sets comprises: traversing each subset of all candidate training data sets, and obtaining the mutual information indicators of the current subset through the following formula: in, For the current subset Q Mutual information indicators The entropy of the data features of the current subset. The entropy of the data label information of the current subset. The information entropy is the joint value of features and labels of the current subset. the step of taking the mutual information indicators as evaluation indicators of a Shapley formula, and obtaining Shapley values of each candidate training data set through the Shapley formula comprises: obtaining the Shapley values of each candidate training data set through the following formula: in, For the first i The Shapley values ​​of the candidate training datasets. The total number of candidate training datasets. For a subset of all candidate training datasets, for The number of candidate training datasets included. For the first i One candidate training dataset, The set of all candidate training datasets, for Mutual information indicators for Mutual information indicators. 2.The image classification model training data selection method of claim 1, wherein, the step of obtaining data feature information entropy, data label information entropy and feature and label joint value information entropy of each subset of all candidate training data sets according to the data features and data labels of each training data in each candidate training data set comprises: The data feature information entropy of each subset of all candidate training data sets is obtained by traversing all candidate training data sets, through the following formula : wherein, is a probability of a data feature of a total of i data features of the data features of each training data in the current subset, t is a total number of data features of the data features of each training data in the current subset; traversing all subsets of the candidate training data sets, obtaining data label information entropy of the current subset by the following formula : wherein, is a probability of a data label of a current training data in the current subset, i m is a total number of data labels of the current training data in the current subset;​ traversing all subsets of the candidate training dataset, obtaining a feature and label joint value information entropy of a current subset by the following formula : wherein, is the probability of the feature and label joint value of the current training data in the current subset, i is the probability of the feature and label joint value of the current training data in the current subset, k is the total number of feature and label joint values of the current training data in the current subset, and the feature and label joint value is obtained by appending the data label to the data feature.

3. The image classification model training data selection method of claim 1 or 2, wherein, the data features are discretized before the data features are discretized.

4. The image classification model training data selection method of claim 3, wherein, the step of discretizing the data features comprises discretizing the data features through a quantity-based binning method, an equidistant binning method or a clustering method.

5. An image classification method characterized by, The method comprises the following steps: obtaining an image to be classified, inputting the image to be classified into a preset image classification model, and obtaining an image classification result of the image to be classified; wherein the image classification model is trained based on the image classification model training data selected by the image classification model training data selection method of claim 1.

6. An image classification model training data selection system, comprising: The method comprises the following steps: a data acquisition module is configured to obtain each candidate training data set of an image classification model, and obtain data features and data labels of each training data in each candidate training data set; an information entropy calculation module is configured to obtain data feature information entropy, data label information entropy and feature and label joint value information entropy of each subset of all candidate training data sets according to the data features and data labels of each training data in each candidate training data set; a mutual information calculation module is configured to obtain mutual information indicators of each subset of all candidate training data sets according to the data feature information entropy, data label information entropy and feature and label joint value information entropy of each subset of all candidate training data sets; The sharp value calculation module is configured to take the mutual information index as an evaluation index of the sharp formula, and obtain the sharp value of each candidate training data set by the sharp formula; The selection module is configured to select the first preset candidate training data set as the image classification model training data according to the order from large to small of the sharp values of the candidate training data sets. The mutual information index of each subset of all candidate training data sets is obtained according to the data feature information entropy, the data label information entropy and the feature and label joint value information entropy of each subset of all candidate training data sets, and the mutual information index of each subset of all candidate training data sets is obtained by the following formula: The mutual information index of the current subset is obtained by the following formula by traversing each subset of all candidate training data sets: in, For the current subset Q Mutual information indicators The entropy of the data features of the current subset. The entropy of the data label information of the current subset. The information entropy is the joint value of features and labels of the current subset. The mutual information index is taken as an evaluation index of the sharp formula, and the sharp value of each candidate training data set is obtained by the sharp formula, including: The sharp value of each candidate training data set is obtained by the following formula: wherein, is the Shapley value of the i th candidate training dataset, is the total number of candidate training datasets, is a subset of all candidate training datasets, is the number of candidate training datasets in is the i th candidate training dataset, is the set of all candidate training datasets, is the mutual information metric of is the mutual information metric of .​​ 7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the image classification model training data selection method in any one of claims 1 to 4.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to realize the steps of the image classification model training data selection method in any one of claims 1 to 4.

Citation Information

Patent Citations

  • Feature selection method and device for model training and electronic equipment

    CN111104572A

  • Federal learning-based feature selection method and device

    CN115409202A