Targeted data selection method based on approximate Shapley value, electronic equipment and medium

Abstract

The invention discloses a target data selection method based on an approximate Shapley value, electronic equipment and a medium. The target data selection method comprises the steps of obtaining a ratio of a source data set to a target utility, and calculating a total utility value and a target utility value. And screening to obtain an extended sample and a feature set. And sample optimization: carrying out Monte Carlo sampling on the expanded sample set to generate random arrangement, forming a plurality of third subsets and calculating utility values of the third subsets. And after a sample Shapley value is calculated, selecting a subset with the minimum error to carry out iterative sample replacement. And if the error reduction degree is smaller than a second threshold value, re-sampling, otherwise, taking a subset of which the error is smaller than a third threshold value as an initial candidate. And if the final error still exceeds the tolerance threshold, turning to feature optimization. And feature optimization: the process is similar to sample optimization, and feature iteration and replacement are guided through a Shapley value. And alternately optimizing the sample subsets and the feature subsets until the utility value error of the combined candidate set (the candidate sample subsets + the candidate feature subsets) is smaller than an error tolerance threshold.

Description

Technical Field

[0001] This invention belongs to the field of data valuation and data selection technology, specifically relating to a targeted data selection method, electronic device, and medium based on approximate Shapley values. Background Technology

[0002] Data selection is the process of filtering a subset from a large dataset to meet predefined requirements. Most existing research focuses on obtaining the most useful subset within size constraints, but there is a lack of research on goal-oriented data selection. In many practical applications, data flow is often strictly limited by budget. For example, in data transactions, model training, and resource allocation, data requesters typically face limited budgets for data selection. After the data requester provides their budget requirements, the data provider should translate the budget constraint into a utility objective and perform utility-goal-oriented data selection, delivering the resulting subset to the data requester. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a targeted data selection method, electronic device, and medium based on approximate Shapley values.

[0004] In a first aspect, embodiments of the present invention provide a targeted data selection method based on approximate Shapley values, comprising: Obtain the source data and target utility ratio, and calculate the total utility value and target utility value corresponding to the source dataset; Set an initial interval, and randomly select an initial sample subset from the initial interval of the source data. Randomly select an initial feature subset from the initial data subset. If the current subset If the error between the utility value and the target utility is less than a first threshold, then the current subset is taken as the first subset; the remaining data of the source data excluding the initial interval are divided into several levels according to categories, and sampling is performed at each level to obtain the second subset; the first subset and the second subset are merged. Randomly sample from the remaining features, and... Merging yields an extended feature subset Ultimately, the core set was obtained. .

[0005] The following optimization processes are executed alternately: right Monte Carlo sampling is performed to generate random permutations. Several third subsets are generated based on each permutation, and the utility value of each third subset is calculated. Based on the utility value of each third subset, the following calculations are performed: The Shapley value of each sample is used; the third subset with the smallest error to the target utility is selected as the current candidate subset, based on... The Shapley value of each sample is used to iteratively replace samples in the current candidate subset. If the reduction in error between the utility value of the current candidate subset before and after the iteration is less than a second threshold, Monte Carlo sampling is performed again. If the reduction in error between the utility value of the current candidate subset before and after the iteration is greater than a second threshold, all third subsets whose error with the target utility is less than a third threshold are obtained as the initial candidate subset. Based on... The Shapley value of each sample is used to iteratively replace samples in the initial candidate subset. When the error between the utility value corresponding to the optimized candidate subset and the target utility is less than the error tolerance threshold, the optimized candidate subset is selected as the final subset.

[0006] right Perform Monte Carlo sampling to generate several random feature permutations. Based on each permutation, generate several fourth subsets and calculate the utility value of each fourth subset. Calculate the utility value of each fourth subset. The Shapley value of each feature is used; the third subset with the smallest error to the target utility is selected as the current candidate subset, based on... The Shapley value of each feature is used to iteratively replace features in the current candidate subset. If the reduction in error between the utility value of the current candidate subset before and after the iteration is less than a second threshold, Monte Carlo sampling is performed again. If the reduction in error between the utility value of the current candidate subset before and after the iteration is greater than a second threshold, all third subsets whose error with the target utility is less than a third threshold are obtained as the initial candidate subset. Based on... The Shapley value of each sample is used to iteratively replace the features of the initial candidate subset. When the error between the utility value corresponding to the optimized candidate subset and the target utility is less than the error tolerance threshold, the optimized candidate subset is selected as the final subset.

[0007] In a second aspect, embodiments of the present invention provide an electronic device, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-described targeted data selection method based on approximate Shapley values.

[0008] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the above-described targeted data selection method based on approximate Shapley values.

[0009] Fourthly, embodiments of the present invention provide a computer program product, including a computer program / instruction, characterized in that, when the computer program / instruction is executed by a processor, it implements the above-described targeted data selection method based on approximate Shapley values.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides a targeted data selection method based on approximate Shapley values. By using approximate Shapley values ​​as a guiding indicator for data selection, it quickly locates a subset of data that is close to the target utility. Specifically, this invention constructs a flexible and efficient data circulation mechanism, forming a closed-loop process of "sample optimization → feature optimization → sample re-optimization". This enables data requesters to obtain data within a given budget through subset selection methods, allowing data resources to be allocated fairly under budget constraints, much like a "on-demand" public service. This provides an effective solution to the widespread data selection problem under budget constraints in reality. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart of a targeted data selection method based on approximate Shapley values ​​provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0015] To facilitate understanding of the solutions in the embodiments of this application by those skilled in the art, some concepts and terms involved in the embodiments of this application will be explained below.

[0016] Shapley Value: The Shapley value is a core indicator in cooperative game theory for allocating fair value. Its core objective is to address how to fairly quantify the contribution of each participant when multiple participants collaborate to generate total utility. In the Shapley value system, each participant allocates value according to their marginal contribution within different alliances. Therefore, the Shapley value system can accurately calculate the specific contribution of each participant (sample or feature) to the overall utility, thus providing theoretical support for data value assessment.

[0017] like Figure 1 As shown, this embodiment of the invention provides a targeted data selection method based on approximate Shapley values. The Shapley value is used to evaluate the marginal contribution of samples and features to model performance, thereby achieving accurate two-dimensional data selection. The method specifically includes the following steps: Step S1: Obtain source data D and target utility ratio. Calculate the total utility value U(D) and the target utility value T corresponding to the source dataset. .

[0018] Step S2: Select the extended sample set and extended feature set from the source data; combine the extended sample set and extended feature set into the core set.

[0019] Specifically, step S2 includes the following sub-steps: Step S201: Set the initial sample interval and the initial feature interval.

[0020] Step S202: Sample and search for the first sample subset in the initial sample interval of the source data D. The first feature subset is searched by sampling within the initial feature interval of the source data D. If the error between the utility value of the current subset and the target utility T is less than the first error threshold, then the current subset is taken as the first subset. If a subset that meets the conditions is not found after multiple rounds of sampling, the sampling range is expanded and the iterative search continues.

[0021] Step S203, transfer the remaining samples S The sample is divided into L levels according to category, and sampling quotas are allocated according to the category ratio to ensure category balance. At each level, the most representative samples are selected based on the K-Center algorithm to obtain the second sample subset. The first sample subset With the second sample subset Merged into an expanded sample set , .

[0022] Step S204, in the remaining features Random sampling was performed to obtain the second feature subset. , with the first feature subset Merging yields an extended feature set .

[0023] Step S205: Merge and expand the sample set Extended feature set , obtain core set .

[0024] Step S3, expand the sample set Monte Carlo sampling is performed to generate random permutations. Several third subsets are generated for each permutation, and the utility value of each third subset is calculated. An expanded sample set is then calculated based on the utility value of each third subset. The Shapley value for each sample.

[0025] The process of generating several third subsets based on each permutation includes: if a random permutation is {s1, s2, s3, ..., s...} n}, {s1,}, {s1,s2,}, {s1,s2,s3},…, {s1,s2,s3,…, s n {0, 1, ..., n} are used as the third subset. For example, the process of generating several third subsets based on each permutation includes: assuming a permutation is {0, 1, ..., n}, then {0}, {0, 1}, {0, 1, 2}, ..., {0, 1, ..., n} are used as the third subsets.

[0026] Step S4, Sample Subset Trial Adjustment Stage: Select the third subset with the smallest error to the target utility as the current candidate subset. Iterate and replace the samples of the current candidate subset based on the Shapley value of each sample in the core set. If the error between the utility value of the current candidate subset before and after the iteration and replacement and the target utility is reduced to less than the second threshold, then Monte Carlo sampling is performed again.

[0027] Specifically, step S4 includes: Step S401: Select the third subset with the smallest error to the target utility as the current candidate subset S, for example, .

[0028] Step S402: Based on the Shapley value of each sample in the core set, iteratively replace the samples in the current candidate subset S to obtain the iteratively adjusted current candidate subset S*; if the degree of error reduction between the utility value of the current candidate subset before and after iteration and the target utility is greater than the second threshold, for example, the degree of error reduction... If the error between the utility value of the current candidate subset and the target utility is reduced to less than the second threshold before iteration and replacement, then return to step S3 to perform Monte Carlo sampling again.

[0029] Specifically, in this example, a binary search method is used to iteratively replace samples in the current candidate subset S, or a utility prediction is used to iteratively replace samples in the current candidate subset S.

[0030] The process of iteratively replacing samples in the current candidate subset S using a binary search method includes: Step S40201A, the core set Each sample is sorted in ascending order according to its Shapley value to obtain the first sequence.

[0031] Step S40202A: Obtain the Shapley value corresponding to the sample x to be replaced in the current candidate subset S. Divide the first sequence into a first half interval with a smaller Shapley value and a second half interval with a larger Shapley value according to the Shapley value. If the utility value of the current candidate subset S is greater than the target utility value T, i.e., U(S,F)>T, then perform search and replacement in the first half interval; otherwise, perform search and replacement in the second half interval.

[0032] The process of iteratively replacing samples in the current candidate subset S based on utility prediction includes: Step S40201B: Train a linear regression model based on the utility values ​​of all third subsets. This linear regression model can accurately predict the utility value of a subset. Here, d represents any sample in the current candidate subset, and S represents the current candidate subset. This represents the weight corresponding to sample d. This represents the Shapley value corresponding to sample d. This indicates a paranoid tendency. , It is obtained by fitting during the training process of the linear regression model.

[0033] Step S40202B: For each candidate subset S, calculate its prediction residual. For each sample to be replaced Calculate the ideal compensation value ,in, Indicates sample The corresponding weights Indicates sample The corresponding Shapley value. Based on the core set. For each sample in the core set, the Shapley value is used to search for the sample with the best matching compensation value to replace the sample. .

[0034] Step S40203B: Store the utility records of the candidate subsets obtained during the adjustment process. When the number of newly added records reaches k, perform an incremental update on the utility prediction function. Record and return the candidate subset S* with the lowest error obtained during the adjustment process.

[0035] Step S5, Initial Candidate Sample Subset Selection Stage: When the error between the utility value of the current candidate subset before and after the iteration and the target utility is reduced to a degree greater than the second threshold, all third subsets whose error with the target utility is less than the third threshold are obtained as the initial candidate subsets.

[0036] Specifically, the third subset containing all values ​​whose error to the target utility is less than the third threshold is selected as the initial candidate subset S, i.e. .

[0037] Furthermore, no subset meeting the requirements was found. The Shapley values ​​of each sample were then sorted in descending order to obtain the second sequence. , This represents the nth sample. Represents the nth sample The corresponding Shapley value; a candidate subset is constructed using a greedy strategy, gradually adding samples with the largest Shapley value. When the utility value of the candidate subset exceeds the target utility value, i.e. At this point, a backtracking search is performed, deleting samples from the current subset to reduce the utility value, until the error between the constructed initial candidate subset and the target utility is less than the third threshold, i.e. Return the initial candidate subset This serves as the starting point for subsequent adjustments.

[0038] Step S6, Candidate Sample Subset Adjustment Stage: Based on the Shapley value of each sample in the core set, the initial candidate subset is iteratively replaced. When the error between the utility value corresponding to the optimized candidate subset and the target utility is less than the error tolerance threshold, the optimized candidate subset is taken as the final selected subset (S', F); otherwise, the candidate sample subset S' obtained in the current iteration step is fixed, and steps S7 to S10 are executed to optimize the candidate feature subset.

[0039] Specifically, iterative replacement of candidate subsets based on the Shapley value of each sample in the core set includes: iterative replacement of candidate subsets using a binary search method, or iterative replacement of candidate subsets based on utility prediction.

[0040] The process of iteratively replacing samples in the candidate subset using the binary search method includes: Arrange each sample in the core set in ascending order according to its Shapley value to obtain the first sequence; Obtain the Shapley value corresponding to the sample x to be replaced in the candidate subset. Divide the first sequence into a first half interval with a smaller Shapley value and a second half interval with a larger Shapley value according to the Shapley value. If the utility value of the current candidate subset S is greater than the target utility value T, then search and replace in the first half interval; otherwise, search and replace in the second half interval.

[0041] The process of iteratively replacing samples in the candidate subset based on utility prediction includes: A linear regression model is trained based on the utility values ​​of all third subsets. Where d represents any sample of the current candidate subset, and S represents the current candidate subset. This represents the weight corresponding to sample d. This represents the Shapley value corresponding to sample d; For each candidate subset, calculate its corresponding prediction residual. ; For the samples to be replaced in the candidate subset Calculate the ideal compensation value ;in, Indicates sample The corresponding weights Indicates sample The corresponding Shapley value; Based on the Shapley value of each sample in the core set, search for the sample in the core set that best matches the compensation value to replace the sample. .

[0042] Step S7, expand the feature set Monte Carlo sampling is performed to generate random permutations. Several fourth subsets are generated based on each permutation, and the utility value of each fourth subset is calculated. An expanded feature set is then calculated based on the utility value of each fourth subset. The Shapley value for each feature.

[0043] The process of generating several fourth subsets based on each permutation includes: if a random permutation is {f1, f2, f3, ..., f...} n}, {f1,}, {f1,f2,}, {f1,f2,f3},…, {f1,f2,f3,…, f n {0, 1, ..., n} are used as the third subset. For example, the process of generating several third subsets based on each permutation includes: assuming a permutation is {0, 1, ..., n}, then {0}, {0, 1}, {0, 1, 2}, ..., {0, 1, ..., n} are used as the third subsets.

[0044] Step S8, Feature Subset Trial Adjustment Stage: Select the fourth subset with the smallest error to the target utility as the current candidate feature subset. Based on the Shapley value of each feature in the core set, iteratively replace the features of the current candidate feature subset. If the degree of reduction in error between the utility value of the current candidate feature subset before and after the iterative replacement and the target utility is less than the second threshold, then Monte Carlo sampling is performed again.

[0045] Specifically, step S8 includes: Step S801: Select the fourth subset with the smallest error to the target utility as the current candidate feature subset F; Step S802: Based on the Shapley value of each feature in the core set, iteratively replace the features of the current candidate feature subset F to obtain the iteratively adjusted current candidate feature subset F*; if the degree of reduction in error between the utility value of the current candidate feature subset before and after iteration and the target utility is greater than the second threshold, then execute step S8 initial candidate feature subset selection stage; if the degree of reduction in error between the utility value of the current candidate feature subset before and after iteration and the target utility is less than the second threshold, then return to step S7 to re-perform Monte Carlo sampling.

[0046] Specifically, in this example, a binary search method is used to iteratively replace features of the current candidate feature subset F, or a utility prediction-based method is used to iteratively replace features of the current candidate feature subset F. The process of iteratively replacing features of the current candidate feature subset F using a binary search method includes: Step S80201A: Sort each feature in the core feature set in ascending order according to its Shapley value to obtain the fourth sequence; Step S80202A: Obtain the Shapley value corresponding to the feature f to be replaced in the current candidate feature subset F. Divide the fourth sequence into a first half interval with a smaller Shapley value and a second half interval with a larger Shapley value according to the Shapley value. If the utility value of the current candidate feature subset F is greater than the target utility value T, i.e., U(S,F)>T, then perform search and replacement in the first half interval; otherwise, perform search and replacement in the second half interval.

[0047] The process of iteratively replacing features in the current candidate feature subset F based on utility prediction includes: Step S80201B: Train a linear regression model based on the utility values ​​of all fourth subsets. This linear regression model can accurately predict the utility values ​​of the feature subsets. Step S80202B: For each candidate feature subset F, calculate its prediction residual, and for each feature to be replaced, calculate the ideal compensation value. Step S80203B: Store the utility records of the candidate feature subsets obtained during the adjustment process. When the number of new records reaches k, perform incremental updates on the utility prediction function, record the candidate feature subset F with the lowest error obtained during the adjustment process, and return it.

[0048] Step S9, Initial Candidate Feature Subset Selection Stage: When the error between the utility value of the current candidate feature subset before and after the iteration replacement and the target utility is reduced to a degree greater than the second threshold, obtain all fourth subsets whose error with the target utility is less than the third threshold as the initial candidate feature subset.

[0049] Specifically, the fourth subset of all features whose error with the target utility is less than the third threshold is selected as the initial candidate feature subset F.

[0050] Furthermore, if no suitable feature subset is found, the Shapley values ​​of each feature are sorted in descending order to obtain the fourth sequence. A candidate feature subset is constructed using a greedy strategy, gradually adding the feature with the largest Shapley value. When the utility value of the candidate feature subset is greater than the target utility value, a backtracking search is performed to delete features in the current feature subset to reduce the utility value, until the error between the constructed initial candidate feature subset and the target utility is less than the third threshold. The initial candidate feature subset is then returned as the starting point for subsequent adjustments.

[0051] Step S10, Candidate Feature Subset Adjustment Stage: Based on the Shapley value of each feature in the core feature set, the initial candidate feature subset is iteratively replaced. When the error between the utility value corresponding to the iteratively optimized candidate feature subset and the target utility is less than the error tolerance threshold, the iteratively optimized candidate feature subset is taken as the final selected feature subset (S', F'); otherwise, the candidate feature subset F' obtained in the current iteration step is fixed, and steps S3 to S6 are executed to optimize the candidate sample subset.

[0052] Specifically, iterative replacement of candidate feature subsets based on the Shapley value of each feature in the core feature set includes: iterative replacement of candidate feature subsets using a binary search method, or iterative replacement of candidate feature subsets based on utility prediction.

[0053] The process of iteratively replacing features in the candidate feature subset using the binary search method includes: arranging each feature in the core feature set in ascending order of Shapley value to obtain a third sequence; obtaining the Shapley value corresponding to the feature f to be replaced in the candidate feature subset; dividing the third sequence into a first half interval with a smaller Shapley value and a second half interval with a larger Shapley value based on the Shapley value; if the utility value of the current candidate feature subset F is greater than the target utility value T, then search and replace in the first half interval; otherwise, search and replace in the second half interval.

[0054] The process of iteratively replacing features in candidate feature subsets based on utility prediction includes: training a linear regression model based on the utility values ​​of all fourth subsets; calculating the corresponding prediction residual for each candidate feature subset; calculating the ideal compensation value for the feature to be replaced in the candidate feature subset; and searching for the feature with the best matching compensation value in the core feature set based on the Shapley value of each feature in the core feature set to replace the feature.

[0055] Step S11, and so on, repeat steps S3 to S6 and steps S7 to S10, to alternately optimize the candidate sample subset and the candidate feature subset.

[0056] In summary, this invention provides a targeted data selection method based on approximate Shapley values. By using approximate Shapley values ​​as a guiding indicator for data selection, it quickly locates a subset of data that closely approximates the target utility. Specifically, this invention constructs a flexible and efficient data circulation mechanism, enabling data demanders to acquire data within a given budget through subset selection. This allows data resources to be allocated fairly under budget constraints, much like a "on-demand" public service, providing an effective solution to the widespread data selection problem under budget constraints in reality.

[0057] like Figure 2 As shown, this application provides an electronic device including a memory 101 for storing one or more programs and a processor 102. When the one or more programs are executed by the processor 102, they implement the method as described in any of the first aspects above.

[0058] The system also includes a communication interface 103. The memory 101, processor 102, and communication interface 103 are electrically connected directly or indirectly to each other to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used for signaling or data communication with other node devices.

[0059] The memory 101 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0060] The processor 102 can be an integrated circuit chip with signal processing capabilities. The processor 102 can be a general-purpose processor 102, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0061] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can also be implemented in other ways. The method and system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0062] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0063] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program thereon. When executed by processor 102, the computer program implements the methods described in any of the first aspects above. If the functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0064] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only.

[0065] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A targeted data selection method based on approximate Shapley values, characterized in that, The method includes: Obtain the source data and target utility ratio, and calculate the total utility value and target utility value corresponding to the source dataset; The expanded sample set and expanded feature set are obtained by filtering from the source data; Optimizing the candidate sample subset includes: performing Monte Carlo sampling on the expanded sample set to generate a random sample permutation, thereby generating several third subsets, and calculating the utility value of each third subset; calculating the Shapley value of each sample based on the utility value; selecting the third subset with the smallest error to the target utility as the current candidate subset, and iteratively replacing samples in the current candidate subset based on the Shapley value of each sample. If the reduction in error between the utility value of the current candidate subset before and after the iteration and the target utility is less than a second threshold, then Monte Carlo sampling is performed again; otherwise, all third subsets with errors to the target utility less than a third threshold are obtained as initial candidate subsets; iteratively replacing samples in the initial candidate subset based on the Shapley value of each sample, and if the error between the utility value corresponding to the candidate subset and the target utility is greater than the error tolerance threshold, then the candidate feature subset is optimized. Optimizing the candidate feature subset includes: performing Monte Carlo sampling on the extended feature set to generate a random feature permutation, thereby generating several fourth subsets, and calculating the utility value of each fourth subset; calculating the Shapley value of each sample based on the utility value; selecting the fourth subset with the smallest error to the target utility as the current candidate subset; iteratively replacing the features of the current candidate subset based on the Shapley value of each sample; if the reduction in error between the utility value of the current candidate subset before and after the iterative replacement and the target utility is less than a second threshold, then Monte Carlo sampling is performed again; otherwise, all fourth subsets with errors to the target utility less than a third threshold are obtained as initial candidate subsets; iteratively replacing the features of the initial candidate subset based on the Shapley value of each sample; if the error between the utility value corresponding to the candidate subset and the target utility is greater than the error tolerance threshold, then the candidate sample subset is optimized. By analogy, the candidate sample subset and candidate feature subset are optimized alternately until the error between the utility value corresponding to the optimized candidate subset and the target utility is less than the error tolerance threshold. The candidate subset = (candidate sample subset, candidate feature subset).

2. The targeted data selection method based on approximate Shapley values ​​according to claim 1, characterized in that, The process of filtering from the source data to obtain the expanded sample set and expanded feature set includes: Set the initial sample interval and initial feature interval; In the initial sample interval of the source data, sample and search for the first sample subset. In the initial feature interval of the source data, sample and search for the first feature subset. If the error between the utility value of the current subset and the target utility is less than the first error threshold, then the current subset is taken as the first subset. First subset = (first sample subset, first feature subset). The remaining samples are divided into L levels according to categories, and sampling is performed at each level to obtain the second sample subset; the first sample subset and the second sample subset are combined into an extended sample set. Randomly sample from the remaining features to obtain a second feature subset; combine the first feature subset with the second feature subset to obtain an extended feature set.

3. The targeted data selection method based on approximate Shapley values ​​according to claim 1, characterized in that, The process of generating several third subsets based on each permutation includes: If a random permutation is {s1, s2, s3, ..., s...} n }, {s1,}, {s1,s2,}, {s1,s2,s3},…, {s1,s2,s3,…,s n } as the third subset.

4. The targeted data selection method based on approximate Shapley values ​​according to claim 1, characterized in that, Iterative replacement of samples or features in the candidate subset based on the Shapley value of each sample or feature in the core set includes: The binary search method is used to iteratively replace samples or features of the candidate subset, or the sample or feature of the candidate subset is iteratively replaced based on utility prediction.

5. The targeted data selection method based on approximate Shapley values ​​according to claim 4, characterized in that, The process of iteratively replacing samples or features in a candidate subset using a binary search method includes: Arrange each sample or feature in the core set in ascending order of Shapley value to obtain the first sequence; Obtain the Shapley value corresponding to the sample or feature x to be replaced in the candidate subset. Divide the first sequence into a first half interval with a smaller Shapley value and a second half interval with a larger Shapley value according to the Shapley value. If the utility value of the current candidate subset S is greater than the target utility value T, then search and replace in the first half interval; otherwise, search and replace in the second half interval.

6. The targeted data selection method based on approximate Shapley values ​​according to claim 4, characterized in that, The process of iteratively replacing samples or features in a candidate subset based on utility prediction includes: A linear regression model is trained based on the utility values ​​of all third subsets. Where d represents any sample or feature of the current candidate subset, and S represents the current candidate subset. This represents the weight corresponding to the sample or feature d. This represents the Shapley value corresponding to d; For each candidate subset, calculate its corresponding prediction residual. ; For the samples or features to be replaced in the candidate subset Calculate the ideal compensation value ;in, Represents a sample or feature The corresponding weights express The corresponding Shapley value; Based on the Shapley value of each sample or feature in the core set, search the core set for the sample or feature that best matches the compensation value to replace it. .

7. The targeted data selection method based on approximate Shapley values ​​according to claim 1, characterized in that, When the reduction in error between the utility value of the current candidate subset and the target utility before and after the iterative replacement is greater than the second threshold, the process of obtaining all third subsets whose error with the target utility is less than the third threshold as the initial candidate subset also includes: If no initial candidate subset that meets the requirements is found, the Shapley values ​​of each sample or feature are sorted in descending order to obtain the second sequence. A candidate subset is constructed using a greedy strategy, and samples or features with the largest Shapley values ​​are added step by step. When the utility value of the candidate subset is greater than the target utility value, a backtracking search is performed to delete samples or features in the current candidate subset to reduce the utility value, until the error between the constructed initial candidate subset and the target utility is less than the third threshold.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more of the computer programs being executed by the at least one processor to enable the at least one processor to perform the targeted data selection method based on approximate Shapley values ​​as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the targeted data selection method based on approximate Shapley values ​​as described in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When executed by a processor, the computer program / instruction implements the targeted data selection method based on approximate Shapley values ​​as described in any of claims 1-7.

Images