Sample evaluation and selection method fusing multiple sampling indexes

Abstract

The invention provides a sample evaluation and selection method fusing multiple sampling indexes. Comprising the steps of initializing a selection set, calculating initial information amount values of all samples, removing abnormal value samples, calculating estimated information amount values of unselected samples, calculating representative values of the unselected samples, calculating evaluation values of the samples, gradually selecting the samples based on the evaluation values, and updating the selection set. The information amount is updated by estimating the value of the information amount, the change of the information amount is estimated, a training model does not need to be retrained after a sample is selected every time, the number of data model training times is reduced, samples are selected from different areas of a data set by introducing representative value calculation of the samples, the diversity of the samples is improved, and the accuracy of data model training is improved. Different areas of a data set are covered with a small number of samples, sample redundancy is relieved, a greedy algorithm basis balance is used for calculating cost and selecting quality, and a core sample set which contains high-quality information and has good data distribution coverage is screened out.

Description

Technical Field

[0001] This invention relates to the field of data-driven manufacturing technology, and in particular to a sample evaluation and selection method that integrates multiple sampling indicators. Background Technology

[0002] Existing sample selection methods include those considering a single factor and those considering multiple factors. Single-factor selection methods have limitations, generally selecting samples based on only a single sampling metric. Methods considering representativeness or diversity alone are prone to selecting outliers, while methods considering information value alone result in highly similar samples, leading to sample redundancy and low computational efficiency. For example, the traditional Query-By-Committee (QBC) method requires retraining the committee model after each sample selection, significantly increasing computational resources and time consumption. Existing multi-factor selection methods, such as the Aggregation Value Assessment Method, primarily rely on the Shapley value to calculate sample information value. For datasets containing n samples, Shapley value calculation requires training the model at least n times, resulting in extremely high computational costs. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a sample evaluation and selection method that integrates multiple sampling indicators. By estimating the information value, the information content is updated and changes in information content are predicted. This eliminates the need to retrain the training model after each sample selection, reducing the number of training iterations. By introducing the calculation of the representative value of the samples, samples are selected from different regions of the dataset, improving sample diversity. A small number of samples cover different regions of the dataset, alleviating sample redundancy. A greedy algorithm is used to balance computational cost and selection quality, selecting a core sample set that contains high-quality information and has good data distribution coverage.

[0004] To solve the above-mentioned technical problems, the technical solution used in this invention is as follows:

[0005] The present invention discloses a sample evaluation and selection method that integrates multiple sampling indicators, comprising the following steps:

[0006] A1. Initialize the selection set S.

[0007] A2. Calculate the initial information value of all samples.

[0008] A3. Remove outlier samples.

[0009] A4. Calculate the estimated information value after the information content of the unselected sample changes.

[0010] A5. Calculate the representative value of the unselected sample.

[0011] A6. Calculate the evaluation value of the sample by estimating the information value and representativeness value.

[0012] A7. Use a greedy algorithm to select samples step by step based on the evaluation value.

[0013] A8. Update the selected set based on the selected samples. , The best sample selected in each round.

[0014] A9. Repeat A4-A8 until the number of samples selected meets the preset value.

[0015] Preferably, in A2, the method for calculating the initial information value is as follows:

[0016]

[0017] The sample to be evaluated; L is the number of committee models; It is the first Each model is to be evaluated on the sample The prediction results; It is the average of the predictions from all models; Sample to be evaluated The initial information value.

[0018] Preferably, the removal of outlier samples specifically includes the following steps:

[0019] A3.1 Calculation

[0020]

[0021] A3.2, Information value of all samples minus The minimum value in.

[0022] Preferably, in A4, the method for calculating the estimated information value is as follows:

[0023]

[0024]

[0025]

[0026] To determine the other samples to be evaluated after the j-th selection of the sample to be evaluated. The degree of information value reduction; in the initial state, the degree of information value reduction for all samples to be evaluated is 0. For Gaussian kernel function, Let be the variance of the Gaussian kernel function; This indicates the sample after the j-th selection of the sample to be evaluated. The estimated information value.

[0027] Preferably, in A5, the method for calculating the representative value is as follows:

[0028]

[0029]

[0030] This indicates that when selecting the sample to be evaluated for the j-th time, the sample to be evaluated is... The representative value; Relu(X) is a linear rectified function that ensures that the incremental reduction in information value is greater than or equal to 0.

[0031] Preferably, A6 is:

[0032]

[0033] To select the sample to be evaluated in the j-th selection, the sample to be evaluated is... The comprehensive evaluation value; α∈ is the weighting parameter that balances the estimated information value and representative value.

[0034] Preferably, α∈[0,1].

[0035] Compared with existing technologies, the main advantages of the sample evaluation and selection method integrating multiple sampling indicators described in this invention are as follows:

[0036] By removing outlier samples, the stability and robustness of the data model are improved, avoiding the inclusion of outliers in the selection set. The information content is updated by calculating the estimated information value of samples, and by predicting changes in information content, samples whose information content rapidly decreases after model updates are avoided from being added to the selection set. This eliminates the need to retrain the model after each sample selection, reducing the number of training iterations and improving sample selection efficiency. Simultaneously, representativeness value is used to represent a specific region in the dataset. By introducing the calculation of representativeness value, samples are selected from different regions of the dataset, increasing sample diversity. While ensuring that the selected samples provide high information content, a small number of samples cover different regions of the dataset, alleviating sample redundancy and avoiding the selection of many highly similar but uncertain samples. Then, the estimated information value and representativeness value are combined to calculate the sample evaluation value, and a greedy algorithm is used to progressively select samples, balancing computational cost and selection quality. This allows for the stable and efficient selection of a core sample set from massive amounts of data that contains high-quality information and has good data distribution coverage.

[0037] This method iteratively updates the estimated information value and can select suitable samples based on the existing data in the selection set, making it widely applicable.

[0038] This method integrates three sampling indicators—information content, diversity, and representativeness—when evaluating samples, overcoming the limitations of considering only a single factor and improving the quality of the selected samples.

[0039] In the outlier sample calculation step, the outlier samples are first inverted and then shifted. The inversion is used to implement a value penalty mechanism to adjust the order of outlier samples, placing them after the normal samples. This allows for the selection of normal samples and effectively eliminates outlier samples. Then, the shift is used to perform mathematical transformations to adjust the information value of all samples to a non-negative value, so as to meet the input requirements of the subsequent greedy algorithm.

[0040] In the step of calculating the estimated information value, the decay is determined by the Gaussian kernel function, and the degree of decay is adjusted by similarity, which has high accuracy. When updating the information value, the similar regions of the selected samples are identified to avoid repeated sampling in similar regions in subsequent processes, thereby improving efficiency.

[0041] In the step of calculating representativeness value, representativeness value is related to the estimated information value; representativeness is defined based on dynamically decaying information value, further avoiding the selection of similar samples.

[0042] In the step of calculating the evaluation value of a sample, the weights of information content and representativeness are balanced, and both the information content and representativeness of the sample are considered when selecting a sample.

[0043] Compared with existing technologies, the sample evaluation and selection method of this invention, which integrates multiple sampling indicators, overcomes the shortcomings of traditional active learning strategies (such as QBC) that are prone to selecting outliers and have high sample redundancy. At the same time, its computational complexity is lower than that of the Shapley value. While reducing computational costs, it improves the quality of sample selection. Attached Figure Description

[0044] The above and other objects, features, and advantages of the invention will become clearer through a more detailed description of the preferred embodiments illustrated in the accompanying drawings. The same reference numerals denote the same parts throughout the drawings, and the drawings are not intentionally drawn to scale with actual dimensions; the focus is on illustrating the gist of the invention.

[0045] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0046] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In this embodiment, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the present invention and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0047] It should be noted that when one element is considered to be "connected" to another element, it can be directly connected to and integrated with the other element, or there may be an intervening element present. The terms "mounted," "one end," "the other end," and similar expressions used in this invention are for illustrative purposes only.

[0048] like Figure 1 As shown, a sample evaluation and selection method integrating multiple sampling indicators includes the following steps:

[0049] A1. Initialize the selection set S.

[0050] A2. Calculate the initial information value of all samples.

[0051] A3. Remove outlier samples.

[0052] A4. Calculate the estimated information value after the information content of the unselected sample changes.

[0053] A5. Calculate the representative value of the unselected sample.

[0054] A6. Calculate the evaluation value of the sample by estimating the information value and representativeness value.

[0055] A7. Use a greedy algorithm to select samples step by step based on the evaluation value.

[0056] A8. Update the selected set based on the selected samples. , The best sample selected in each round.

[0057] A9. Repeat A4-A8 until the number of samples selected meets the preset value.

[0058] The above method improves the stability and robustness of the data model by removing outlier samples, thus avoiding their inclusion in the selection set. It updates the information content by calculating the estimated information value of samples, and by predicting changes in information content, it avoids adding samples whose information content rapidly decreases after model updates to the selection set. This eliminates the need to retrain the model after each sample selection, reducing the number of training iterations and improving sample selection efficiency. Simultaneously, the representativeness value is used to represent a specific region in the dataset. By introducing the calculation of the representativeness value, samples are selected from different regions of the dataset, increasing sample diversity. While ensuring that the selected samples provide high information content, a small number of samples cover different regions of the dataset, alleviating sample redundancy and avoiding the selection of many highly similar but uncertain samples. Then, the estimated information value and representativeness value are combined to calculate the sample evaluation value, and a greedy algorithm is used to progressively select samples, balancing computational cost and selection quality. This method stably and efficiently filters a core sample set from massive datasets that contains high-quality information and has good data distribution coverage. This method, by iteratively updating the estimated information value, can select suitable samples based on the existing data in the selection set, making it widely applicable.

[0059] In the above methods:

[0060] In A2, the initial information value is calculated as follows:

[0061]

[0062] The sample to be evaluated; L is the number of committee models; It is the first Each model is to be evaluated on the sample The prediction results; It is the average of the predictions from all models; Sample to be evaluated The initial information value. In this embodiment, the committee model is obtained by training on a randomly sampled subset of labeled data; when predicting continuous values, such as regression problems, the predicted values ​​are used directly; when predicting discrete categories, such as classification problems, the predicted probabilities of each category are used.

[0063] The removal of outlier samples specifically includes the following steps:

[0064] A3.1 Calculation

[0065]

[0066] A3.2, Information value of all samples minus Find the minimum value in the range; perform translation adjustments on all samples.

[0067] The outlier samples are those with high initial information value. This is the inverse value of the outlier sample;

[0068] The above method first inverts the outlier samples and then performs a translation. The inversion is used to implement a value penalty mechanism, adjusting the order of outlier samples so that they are placed after normal samples. This allows normal samples to be selected first, effectively eliminating outlier samples. Then, the translation is used to perform mathematical transformations to adjust the information value of all samples to a non-negative value, so as to meet the input requirements of the subsequent greedy algorithm.

[0069] The following are examples:

[0070] Table 1 compares the inversion and shifting processes performed on some outlier samples and non-outlier samples.

[0071]

[0072] Table 1

[0073] Taking Table 1 as an example, in one embodiment, the shift adjustment is the current information value minus -20; among the samples A, B, C, D, and E above, samples E and C are outlier samples; samples A, B, and D are all normal samples. Based on the initial information value, the samples are sorted as follows: E > C > A > B > D.

[0074] By performing the inversion process, the sorting is adjusted to A(10) > B(8) > D(5) > C(-15) > E(-20); outlier samples are marked as range samples; the outlier sample marking is sorted last; based on the information value, the samples selected earlier are given priority, thus effectively excluding outlier samples.

[0075] By performing a translation process, the information value of each sample is reduced by -20, and the sorting is adjusted to A(30) > B(28) > D(25) > C(5) > E(0). The information value of all samples is adjusted to a non-negative value by the translation process, which satisfies the input requirements of the subsequent greedy algorithm.

[0076] In A4, the method for calculating the estimated information value is as follows:

[0077]

[0078]

[0079]

[0080] To determine the other samples to be evaluated after the j-th selection of the sample to be evaluated. The degree of information value reduction; in the initial state, the degree of information value reduction for all samples to be evaluated is 0. For Gaussian kernel function, Let be the variance of the Gaussian kernel function; This indicates the sample after the j-th selection of the sample to be evaluated. The estimated information value. For the newly selected sample.

[0081] By introducing a cumulative attenuation factor Whenever a new sample is selected All other samples All will be based on and The similarity is used to cumulatively reduce the future information value. In this embodiment, the measurement is performed using a kernel function. and Similarity. By introducing a decay factor. This allows for forward-looking calculation of estimated information value, proactively avoiding repeated sampling in similar areas and alleviating the problem of sample redundancy.

[0082] Meanwhile, the decay is determined by the Gaussian kernel function, through... and The similarity adjustment attenuation level is highly accurate. (Compared to selected samples) More similar samples The closer the kernel function is to 1, the lower its information value; and The more similar they are, the more severe the attenuation. and The greater the difference, the smaller the attenuation.

[0083] The above method identifies similar regions of selected samples when updating information value, avoiding repeated sampling in similar regions in subsequent samples and improving efficiency.

[0084] In A5, the method for calculating representative value is as follows:

[0085]

[0086]

[0087] This indicates that when selecting the sample to be evaluated for the j-th time, the sample to be evaluated is... The representative value; Relu(X) is a linear rectified function that ensures that the incremental reduction in information value is greater than or equal to 0.

[0088] Sample to be evaluated The information content of other samples is reduced incrementally; when calculating the sample to be evaluated When assessing the representative value, the sample to be evaluated should be taken into account. Relationship with other samples in the dataset The loop variable for the sample.

[0089] The above methods represent value Based on attenuation factor and the value of initial information The calculation makes the representative value related to the estimated information value; the representativeness is defined based on the dynamically decaying information value, which further avoids the selection of similar samples.

[0090] A6 specifically refers to:

[0091]

[0092] To select the sample to be evaluated in the j-th selection, the sample to be evaluated is... The comprehensive evaluation value; α∈ is the weighting parameter that balances the estimated information value and representative value. α∈[0,1]; in a preferred embodiment, α=0.5.

[0093] This represents the estimated information content after dynamic decay. A weighting parameter α is used to balance the information content and representativeness of the samples; when α is close to 1, samples with high information content are preferred; when α is close to 0, samples with good representativeness are preferred; in this embodiment, α = 0.5, balancing information content and representativeness when selecting samples. Samples to be evaluated An evaluation will be conducted. Then, based on each... The evaluation values ​​were selected from the sample.

[0094] This method integrates three sampling metrics—information content, diversity, and representativeness—when evaluating samples, overcoming the limitations of considering only a single factor and improving the quality of selected samples. It overcomes the shortcomings of traditional active learning strategies (such as QBC) that easily select outliers and have high sample redundancy, while having lower computational complexity than the Shapley value. This reduces computational costs while training a machine learning model with superior performance and stronger robustness.

[0095] The correlation coefficients between this method and the traditional QBC method for sample selection order are 0.70 (p-value 1.03e-68) on the composite solidified dataset; 0.50 (p-value 5.85e-36) on the milling stability dataset; and 0.83 (p-value 5.21e-167) on the rolling bearing failure dataset. This method effectively simulates the selection behavior of traditional QBC, and only requires training the committee model in the initial state, avoiding repeated model training and significantly improving computational efficiency.

[0096] The correlation coefficients between our method and the Shapley value calculation method regarding sample selection order are 0.32 (p=8.96e-10) on the composite solidification dataset; 0.44 (p=4.40e-12) on the milling stability dataset; and 0.58 (p=6.92e-39) on the rolling bearing failure dataset. This method can serve as an alternative to the Shapley value calculation method for evaluating sample information content, while avoiding the high computational cost of alternative methods, making it suitable for handling large-scale datasets.

[0097] In this specification, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0098] In the description of this specification, the references to terms such as "preferred embodiment," "another embodiment," "other embodiment," or "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0099] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A sample evaluation and selection method fusing multiple sampling indicators, characterized in that: The method comprises the following steps: A1, initializing a selection set S; A2, calculating initial information quantity values of all samples; A3, removing outlier samples; A4, calculating estimated information quantity values of unselected samples after information quantity change; A5, calculating representative values of unselected samples; A6, calculating evaluation values of samples by the estimated information quantity values and the representative values; A7, gradually selecting samples based on the evaluation values using a greedy algorithm; A8. updating the selection pool based on the selected samples, best sample selected for each round;​ A9, repeating A4-A8 until the number of selected samples meets a preset value.

2. The method according to claim 1, wherein: In A2, the calculation method of the initial information quantity values is as follows: The sample to be evaluated; L is the number of committee models; It is the first Each model is to be evaluated on the sample The prediction results; It is the average of the predictions from all models; Sample to be evaluated The initial information value. 3.The method of claim 1, wherein the method further comprises: The removing of the outlier samples specifically comprises the following steps: A3.1, computing A3.2, the information value of all samples minus the minimum value in.

4. The method according to claim 1, wherein: In A4, the calculation method of the estimated information quantity values is as follows: is the information value reduction degree of the other samples to be evaluated after the jth selection of the sample to be evaluated; the information value reduction degree of all the samples to be evaluated in the initial state is 0; is the information value reduction degree of the other samples to be evaluated after the jth selection of the sample to be evaluated; the information value reduction degree of all the samples to be evaluated in the initial state is 0; is a Gaussian kernel function, is a variance of the Gaussian kernel function; represents the estimated information value of the sample after the jth selection of the sample to be evaluated.

5. The method according to claim 1, wherein: In A5, the calculation method of the representative values is as follows: represents the representative value of the sample to be evaluated at the jth selection of the sample to be evaluated ; Relu(X) is a linear rectifier function that ensures that the information quantity value reduction increment is greater than or equal to 0.

6. The method according to claim 1, wherein: A6 is specifically as follows: To select the sample to be evaluated in the j-th selection, the sample to be evaluated is... The comprehensive evaluation value; α∈ is the weighting parameter that balances the estimated information value and representative value.

7. The method according to claim 6, wherein: 。

Images