A sample screening method, electronic device, storage medium and program product

Abstract

The application discloses a sample screening method, an electronic device, a storage medium and a program product; the method comprises the following steps: acquiring a candidate sample set, wherein the candidate sample set comprises at least one candidate sample; scoring the classification generalization attribute of each candidate sample to determine the score corresponding to the candidate sample, wherein the score corresponding to the candidate sample is at least one, and the scoring mode of each score is different; the scoring mode comprises the supplementary contribution degree of the sample content and the classification label of the candidate sample to the training sample set, and the distinguishing contribution degree of the word level belonging classification in the sample content of the candidate sample; processing the score corresponding to each candidate sample to determine the target score of the candidate sample; screening each candidate sample based on the target score of the candidate sample to obtain at least one target sample; the problems of high sample screening cost and imbalance are solved; the sample is effectively screened, and the model iteration efficiency and prediction robustness are improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a sample screening method, electronic device, storage medium, and program product. Background Technology

[0002] With the development of technology, neural network models are being used more and more widely in various industries. Before use, models need to be trained, which requires a large number of training samples. Collecting a large number of training samples is time-consuming and labor-intensive. To save time and effort, a method of sample generalization has been proposed to expand the training samples. However, the generalized samples need to be selected to be suitable for model training. Currently, the selection of sample sets is mainly done manually or experimentally (model training, performance verification), but this is costly and prone to sample imbalance, thus affecting the model's generalization ability. Summary of the Invention

[0003] This invention provides a sample screening method, electronic device, storage medium, and program product to solve problems such as high sample screening costs and imbalance.

[0004] According to one aspect of the present invention, a sample screening method is provided, comprising:

[0005] Obtain a candidate sample set, wherein the candidate sample set includes at least one candidate sample;

[0006] For each candidate sample, the classification generalization attribute of the candidate sample is scored to determine the score corresponding to the candidate sample. There is at least one score corresponding to the candidate sample, and each score is scored in a different way. The scoring method includes the supplementary contribution of the candidate sample's sample content and classification label to the training sample set, as well as the distinguishing contribution of the word level classification in the candidate sample's sample content.

[0007] For each candidate sample, the score corresponding to the candidate sample is processed to determine the target score of the candidate sample;

[0008] Based on the target scores of the candidate samples, each candidate sample is screened to obtain at least one target sample.

[0009] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0010] At least one processor, and a memory communicatively connected to said at least one processor;

[0011] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the sample screening method according to any embodiment of the present invention.

[0012] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the sample screening method according to any embodiment of the present invention.

[0013] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the sample screening method described in any embodiment of the present invention.

[0014] The technical solution of this invention involves obtaining a candidate sample set, which includes at least one candidate sample; for each candidate sample, scoring its classification generalization attribute to determine a score, wherein each candidate sample has at least one score, and each score is scored using a different method; the scoring method includes the contribution of the candidate sample's content and classification label to the training sample set, and the distinguishing contribution of the word-level classification in the candidate sample's content; for each candidate sample, processing its score to determine a target score; and filtering the candidate samples based on their target scores to obtain at least one target sample; thus solving the problem of high sample screening costs. This method addresses issues such as high accuracy and imbalance. It employs different scoring methods to score the classification generalization attributes of candidate samples, obtaining at least one score for each candidate sample. By processing multiple scores, a target score for each candidate sample is obtained, and then target samples are selected based on this target score. When scoring candidate samples, the contribution of the sample content and classification label to the training sample set is scored, effectively selecting target samples to enrich the training sample set and improving model generalization performance. Furthermore, the method scores the contribution of words in the sample content to different categories, selecting samples containing more specific features based on the contribution of words to different categories. The sample selection method provided in this application can effectively select samples, improving model iteration efficiency and prediction robustness.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a sample screening method provided in Embodiment 1 of the present invention;

[0018] Figure 2 This is a flowchart of a sample screening method provided in Embodiment 2 of the present invention;

[0019] Figure 3 This is a schematic diagram of the structure of a sample screening device according to Embodiment 3 of the present invention;

[0020] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the sample screening method of this invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] Example 1

[0024] Figure 1This is a flowchart of a sample screening method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations requiring effective sample screening. The method can be executed by a sample screening device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0025] S101. Obtain a candidate sample set, which includes at least one candidate sample.

[0026] In this embodiment, a candidate sample can be understood as a sample with screening requirements, and the candidate sample can be used for model training; the candidate sample set can be understood as a dataset formed by the candidate samples, and the candidate sample set includes one or more candidate samples.

[0027] Candidate samples are pre-generated. These can be generated by generalizing from training samples, forming a candidate sample set based on one or more candidate samples. The number of candidate samples can be determined based on the original number of training samples, the accuracy requirements of model training, the type of model, and the size of the model. During model training, it may be necessary to collect training samples in advance. However, under special conditions or business scenarios, collecting training samples can be difficult or time-consuming. Therefore, the training samples can be generalized to generate a large number of candidate samples; alternatively, a large number of candidate samples can be collected during model training. The obtained candidate samples typically need to be screened to select those suitable for model training or those that positively impact model training. After generating the candidate sample set, it can be stored and retrieved from the corresponding storage space when sample screening is required. The method provided in this application embodiment is used to screen candidate samples to obtain samples suitable for model training. This application embodiment can obtain the candidate sample set when screening conditions are met. The screening conditions can be set according to specific model types, business types, model accuracy requirements, etc., for example, periodically screening candidate samples, screening candidate samples after receiving a user trigger command, etc.

[0028] S102. For each candidate sample, score the classification generalization attribute of the candidate sample to determine the score corresponding to the candidate sample. There is at least one score corresponding to the candidate sample, and the scoring method for each score is different. The scoring method includes the supplementary contribution of the sample content and classification label of the candidate sample to the training sample set, as well as the distinguishing contribution of the word level classification in the sample content of the candidate sample.

[0029] In this embodiment, the classification generalization attribute of a candidate sample refers to its purpose of classification generalization. Different types of models use samples with different attributes / purposes, such as classification, regression, and detection. In this embodiment, the candidate samples are primarily used for model classification generalization; therefore, the classification generalization attribute is scored. The scoring method involves methods and rules for evaluating candidate samples. The scoring method includes the supplementary contribution of the candidate sample's content and classification label to the training sample set, and the discriminative contribution of the words in the candidate sample's content to their respective categories. Supplementary contribution refers to the enrichment of the training sample set after the candidate sample is added; discriminative contribution refers to the role of the words in the candidate sample in the classification result of the candidate sample, that is, the role of the words in the candidate sample in distinguishing the category to which the candidate sample belongs.

[0030] The system pre-calculates the contribution of candidate samples to the training sample set based on their sample content and classification labels, and sets different scoring methods for the distinguishing contribution of words in the sample content to the classification set. One or more scoring methods can be set for both supplementary contribution and distinguishing contribution. For the supplementary contribution scoring method, the influence of the candidate sample content on the classification label is determined based on the sample content and classification label. The supplementary contribution of the candidate sample to the training sample set is determined based on the degree of influence, and the candidate sample is scored based on the supplementary contribution to obtain a corresponding score. For the distinguishing contribution scoring method, the classification of words in the candidate sample content is analyzed, and the influence of the words on different classifications is determined based on the classification category. The distinguishing contribution of words in the sample content to the classification set is determined based on the degree of influence, and the candidate sample is scored based on the distinguishing contribution to obtain a corresponding score. This embodiment of the application can set one or more scoring methods, and score the classification generalization attribute of the candidate sample according to the pre-set scoring methods to determine the score used to evaluate whether the candidate sample can be used for classification generalization. Each scoring method yields a corresponding score, and multiple scores can be obtained when multiple scoring methods are used to score a candidate sample.

[0031] By scoring the contribution of candidate samples to the training sample set based on their content and classification labels, samples different from those already in the training sample set can be selected, enriching the training sample set and improving the model's generalization performance. Furthermore, by evaluating the discriminative contribution of word-level classification within the candidate sample content, samples containing more specific features can be selected, improving the model's training accuracy.

[0032] S103. For each candidate sample, process the score corresponding to the candidate sample to determine the target score of the candidate sample.

[0033] In this embodiment, the target score can be understood as the final score obtained by the candidate sample, which is determined based on all the scores corresponding to the candidate sample. For each candidate sample, all the scores corresponding to the candidate sample are determined, and each score is comprehensively processed, for example, by performing weighted summation, taking the maximum value, or taking the minimum value, to obtain the target score. The same processing method is used for each candidate sample to obtain its corresponding target score.

[0034] S104. Based on the target scores of the candidate samples, each candidate sample is screened to obtain at least one target sample.

[0035] In this embodiment, the target sample can be understood as a sample selected from the candidate samples. The selected target samples can be used for model training to improve the model's generalization ability. The target scores of each candidate sample are compared, and the candidate samples are selected based on the magnitude of the target scores. For example, K candidate samples are selected as target samples in descending order of target scores. The value of K can be preset, set to a fixed value, or dynamically set according to the number of candidate samples. For example, K = number of candidate samples * 80%.

[0036] The sample screening method provided in this invention involves obtaining a candidate sample set, which includes at least one candidate sample; for each candidate sample, scoring its classification generalization attribute to determine a corresponding score, wherein each candidate sample has at least one corresponding score, and each score is scored using a different method; the scoring method includes the supplementary contribution of the candidate sample's sample content and classification label to the training sample set, and the distinguishing contribution of the word-level classification in the candidate sample's sample content; for each candidate sample, processing the corresponding score to determine a target score; and screening each candidate sample based on its target score to obtain at least one target sample; this method solves the problems of high sample screening cost and imbalance. The method employs different scoring methods to score the classification generalization attributes of candidate samples, obtaining at least one score for each candidate sample. By processing multiple scores, a target score for the candidate sample is obtained, and then target samples are selected based on the target score. When scoring candidate samples, the contribution of the candidate sample's content and classification label to the training sample set is scored, which can effectively select target samples to enrich the training sample set and improve the model's generalization performance. The method also scores the contribution of words in the candidate sample content to different categories, selecting samples containing more specific features based on the contribution of words to different categories. The sample selection method provided in this application can effectively select samples, and training the model based on the selected target samples can improve the model's iteration efficiency and prediction robustness.

[0037] Example 2

[0038] Figure 2 This is a flowchart of a sample screening method provided in Embodiment 2 of the present invention. This embodiment is a refinement based on the above embodiments. Figure 2 As shown, the method includes:

[0039] S201. Obtain a candidate sample set, which includes at least one candidate sample.

[0040] S202. For each candidate sample, score the classification generalization attribute of the candidate sample to determine at least one score corresponding to the candidate sample. The scoring method for each score corresponding to the candidate sample is different. The scoring method includes the supplementary contribution of the candidate sample's sample content and classification label to the training sample set, as well as the distinguishing contribution of the word level classification in the candidate sample's sample content.

[0041] The scoring method is the contribution of the candidate sample's content and classification label to the training sample set, which can be achieved through the scoring methods A1-A3; the scoring method is the contribution of the candidate sample's content to the classification of words at the word level, which can be achieved through the scoring methods D1-D4 and E1-E3.

[0042] As an optional embodiment, this optional embodiment will score the classification generalization attribute of the candidate samples to determine the score corresponding to the candidate samples, which is optimized into steps A1-A3:

[0043] A1. Based on the candidate samples, retrieve the training samples in the pre-generated training sample set to determine the recalled samples and the matching scores corresponding to the recalled samples.

[0044] In this embodiment, the recalled sample can be understood as a sample recalled from the training sample set. The training sample set includes training samples, and the number of training samples can be determined according to the model size, accuracy, type, etc. The matching score can be understood as the score obtained by matching the recalled sample with the candidate sample.

[0045] A training sample set is pre-generated, which typically includes a large number of training samples. The training samples are then retrieved based on the candidate samples. For example, the retrieval can be performed directly based on the text content of the candidate samples or based on the text vector of the candidate samples. The matching score between the candidate samples and each training sample is calculated. Based on the matching score, the training samples with higher matching degree are selected as the recall samples, and the matching score corresponding to each recall sample is determined.

[0046] As an optional embodiment, this optional embodiment will retrieve training samples from a pre-generated training sample set based on candidate samples, determine the recalled samples and the matching scores corresponding to the recalled samples, and optimize it to steps B1-B3:

[0047] B1. Vectorize the candidate samples to obtain the candidate sample vector.

[0048] In this embodiment, the candidate sample vector can be understood as a vector obtained by vectorizing the candidate sample. The candidate sample vector can be a one-dimensional vector or a multi-dimensional vector. Candidate samples are usually composed of text statements. A language model can be used to map the text statements into a vector composed of floating-point data to obtain the candidate sample vector, thus realizing the vectorization of the samples.

[0049] B2. Based on the candidate sample vector and the corresponding training sample vector, perform retrieval and matching to obtain the matching score for each training sample.

[0050] In this embodiment, the training sample vector can be understood as the vector obtained after vectorizing the training samples. All training samples are pre-vectorized to obtain a training sample vector corresponding to each training sample. Training samples are typically composed of text statements, which can be mapped into vectors of floating-point data using a language model. To ensure consistency, training samples and candidate samples can be vectorized in the same way. The training sample vectors can be stored in a vector library and retrieved directly from the library when needed. The similarity between the candidate sample vector and each training sample vector is calculated. The similarity can be cosine distance, Euclidean distance, etc., and the similarity score is used as the matching score between the candidate sample and the training sample.

[0051] B3. Filter each training sample based on the matching score to determine the recalled samples and the corresponding matching scores of the recalled samples.

[0052] The training samples are filtered based on their matching scores, and the training samples with higher matching scores are selected as the recall samples. At the same time, the matching scores of the recall samples are determined. The number of recall samples can be preset. For example, if the number of recall samples is preset to N, then the N training samples with the highest matching scores are selected as the recall samples. The number of recall samples can also be set to a non-fixed value. For example, the training samples with matching scores exceeding a preset score threshold are selected as the recall samples, or the training samples with matching scores in the top 80% are selected as the recall samples, and so on.

[0053] A2. Weight each matching score based on the classification labels of the recalled samples and the candidate samples.

[0054] In this embodiment, the classification labels can be categories such as male / female, pedestrian, motor vehicle, and building, which can be classified according to specific business scenarios. The labels of the recalled samples and the classification labels of the candidate samples are analyzed. For example, the labels of the recalled samples and the classification labels of the candidate samples are compared, and the matching scores are weighted based on label consistency to determine the weight of the matching score; or, different weights are selected for different classification labels, and so on.

[0055] As an optional embodiment, this optional embodiment will weight each matching score according to the classification label of the recalled sample and the classification label of the candidate sample, optimized as follows: for each recalled sample, compare whether the classification label of the recalled sample is consistent with the classification label of the candidate sample. If they are consistent, determine that the weight of the matching score corresponding to the recalled sample is -1; if they are inconsistent, determine that the weight of the matching score corresponding to the recalled sample is 1.

[0056] In this embodiment, the matching score of recalled samples with consistent classification labels is assigned a weight of -1, while the matching score of recalled samples with inconsistent classification labels is assigned a weight of 1. Essentially, samples with highly similar content and consistent classification labels are assigned lower scores, while samples with highly similar content but different labels are assigned higher scores. Samples with highly similar content but different labels contain more information, thereby filtering out training samples that are different from those already in the training sample set, supplementing and enriching the training sample set, and improving the model's generalization ability.

[0057] A3. Determine the score corresponding to the candidate sample based on each weighted matching score.

[0058] After weighting, multiple weighted matching scores are obtained for each candidate sample. The weighted matching scores are then combined and calculated, such as by summation or multiplication. Alternatively, a calculation formula can be pre-set, and the weighted matching scores can be substituted into the formula to calculate the score corresponding to the candidate sample, and so on.

[0059] As an optional embodiment, this optional embodiment determines the score corresponding to the candidate sample based on each weighted matching score, optimized into steps C1 and / or C2:

[0060] C1. Sum the weighted scores of each matching sample to obtain the score corresponding to the candidate sample.

[0061] C2. Calculate the ratio of each weighted matching score to the target length, sum all the ratios to obtain the score of the candidate sample, where the target length is determined based on the length of the candidate sample.

[0062] In this embodiment, the target length can be understood as a predetermined length, which is determined based on the length of the candidate sample. The target length may be different for different candidate samples. For example, the target length is the candidate sample length, target length = log(candidate sample length + 1), and so on. The ratio of each weighted matching score to the target length is calculated, and all the ratios are summed to obtain the score of the candidate sample.

[0063] For example, the scoring of candidate samples can be implemented in the following way:

[0064] 1. Vector-based retrieval scoring

[0065] Using the training sample set as the retrieval document and the candidate samples as the query, a list with a score of 0.7 or higher is selected based on vector retrieval, as follows:

[0066] 1) Text vectorization: Using a language model, the text sentences in the candidate samples are mapped into a one-dimensional vector composed of floating-point data to obtain the candidate sample vector.

[0067] 2) Text Vector Retrieval: Vectorize each training sample in the training sample set to obtain training sample vectors and store them in the vector library. Use the cosine distance of the vectors as the filtering criterion to calculate the cosine distance between the candidate sample vector and the training sample vector to obtain the matching score. The higher the matching score, the better the match.

[0068] 3) Matching sample screening: A list of samples with a score of 0.7 or higher was selected as the recall sample set. The results are shown in Table 1.

[0069] Table 1. Vector Retrieval Matching Score Table

[0070] Recall Sample Number Sample categories Vector retrieval matching score Sample 1 Category 1 0.9 Sample 2 Category 2 0.8 Sample 3 Category 3 0.7

[0071] The final score of the candidate sample is calculated based on the matching score:

[0072]

[0073] Where, emb_score is the score of the candidate sample, that is, the final score of the candidate sample; sign i The weight of the i-th matching score; match_score i Let be the matching score of the i-th recalled sample. Let n be the total number of recalled samples.

[0074]

[0075] Example: If the candidate sample belongs to category 1 and 3 samples are recalled, the candidate sample score is calculated as shown in Table 2.

[0076] Table 2 Score Table

[0077] Recall Sample Number Recall Sample Number Sample categories Vector retrieval matching score Candidate Sample Score Sample 1 Category 1 0.9 -0.9 Sample 2 Category 2 0.8 0.8 Sample 3 Category 3 0.7 0.7 Score 0.6＝-0.9+0.8+0.7

[0078] 2. For each candidate sample, a retrieval score based on words / characters.

[0079] A searchable text database is used: a searchable text database is formed based on training samples from the training sample set. The training sample set is used as the document, and candidate samples are used as queries. The open-source text search tool Elasticsearch is used for retrieval, and the matching scores of the candidate samples are provided. Considering the characteristics of documents of varying lengths, the BM25 matching algorithm is used during the search to mitigate the distribution differences caused by the varying document lengths.

[0080] 1) Matching Sample Screening: Samples with a matching score greater than 10 are selected as the recall samples. Examples of matching scores are shown in Table 3.

[0081] Table 3 Text Retrieval Matching Score Table

[0082] Recall Sample Number Sample categories iTerm search match score Sample 1 Category 1 20 Sample 2 Category 2 17 Sample 3 Category 3 11

[0083] 2) Calculate the scores of candidate samples.

[0084]

[0085] Where, iterm_score is the score of the candidate sample; sign i The weight of the i-th match score; n is the total number of recalled samples; iterm_search_score i is the matching score of the i-th recalled sample; log(candidate sample length + 1) is the target length.

[0086] sign i The calculation formula can be found in the formula above, i.e., -1 when the label of the recalled sample is the same as that of the candidate sample, and 1 when they are different.

[0087] This application provides two retrieval methods. When retrieving training samples from a pre-generated training sample set based on candidate samples, retrieval can be performed based on the vectors of the candidate samples or on the words / characters of the candidate samples using the open-source text retrieval tool Elasticsearch. One method can be selected for retrieval, and the corresponding score determined; alternatively, both methods can be selected for retrieval, and two corresponding scores determined.

[0088] As an optional embodiment, this optional embodiment will score the classification generalization attribute of the candidate samples to determine the score corresponding to the candidate samples, and optimize it to D1-D4:

[0089] D1. Collect perturbation samples based on candidate samples and pre-trained classification models to obtain at least one perturbation sample.

[0090] In this embodiment, the classification model can be understood as a model with classification function. The target samples selected in this embodiment can be used to further train the classification model. Perturbation samples can be understood as samples generated after adjusting and changing candidate samples. Word / character segmentation is performed on the candidate samples, and individual words or characters in the sentences of the candidate samples are replaced to obtain perturbation samples. Different words / characters in the candidate samples are replaced, and the same word / character is replaced with different words / characters, etc., to generate new sentences. The newly generated sentences are input into a pre-trained classification model for classification prediction to obtain the corresponding classification and score. The different words / characters in each sentence and the predicted classification and / or score are used as perturbation samples. Candidate samples can also be input into a pre-trained classification model for classification prediction, and the different words / characters in the candidate samples and the predicted classification and / or score are used as perturbation samples.

[0091] For example, the perturbation samples include: (iterm_1,iterm_2,iterm_m,…,socre), where iterm_1,iterm_2,iterm_m represent different features, socre0 is the classification / score, and the features can refer to different words / characters.

[0092] D2. Train the regression model based on the perturbation samples to obtain the weight parameters of the regression model.

[0093] In this embodiment, the regression model is used to describe the relationship between different variables and the result. For example, the regression model is: y = w_1*x1 + w_2*x2 + ... + w_m*x_m, where y is socre, x1-xm correspond to iterm_1, iterm_2, iterm_m respectively, and w_1...w_m are different weight parameters of the regression model.

[0094] The regression model is trained based on the obtained perturbation model. The influence of different characters / words on the classification results is analyzed, i.e., the discriminative contribution of each character / word to its classification. This yields the weight parameters of the regression model, which represent the importance distribution of each character / word in the model. A larger weight parameter indicates a greater influence of that feature on the classification result; features with almost no influence on the classification result can have a weight parameter of 0. The size of m can be pre-determined based on the length of the candidate samples, etc. Specific features affecting the results are pre-defined and formed into a data table. Then, iterm_1, iterm_2, ..., iterm_m are matched with the data table to train the regression model and obtain its weight parameters. The regression model enables the interpretability of the classification model on the candidate samples.

[0095] D3. Based on the absolute value of the weight parameters in the regression model, filter each weight parameter to obtain a preset number of target weight parameters.

[0096] In this embodiment, the preset number can be determined based on one or more information such as the business scenario, classification type, and length of candidate samples. For example, the preset number is 20. The absolute values ​​of each weight parameter are compared, and the preset number of weight parameters are selected as the target weight parameters in descending order.

[0097] D4. Use the variance of each objective weight parameter as the score corresponding to the candidate sample.

[0098] Calculate the variance of each target weight parameter and use this variance as the score for the candidate sample. The larger the variance, the more discriminative each feature is, the more concentrated the reference features are, indicating that the candidate sample has a certain degree of uniqueness.

[0099] For example, this embodiment provides a scoring method: inter_score = Var(setW); where inter_score is the score of the candidate sample, and setW is the set formed by the target weight parameters.

[0100] As an optional embodiment, this optional embodiment will score the classification generalization attribute of the candidate samples to determine the score corresponding to the candidate samples, optimizing it to E1-E3:

[0101] E1. Segment the candidate samples to obtain at least one word to be classified.

[0102] In this embodiment, the word to be classified can be understood as a word with classification requirements; it can be a single character or multiple characters. The candidate sample is segmented into multiple characters / words using a preset algorithm or model, resulting in at least one word to be classified.

[0103] E2. For each word to be classified, determine the probability of the word in each category, calculate the sum of squares of each probability, and take the difference between 1 and the sum of squares as the Gini coefficient of the word to be classified.

[0104] For each word to be classified, count the number of times it appears in the training samples corresponding to each category. Divide the number of times it appears in the training samples of each category by the total number of times it appears in the training samples of all categories to obtain the probability of the word in that category. Calculate the square of each probability, then sum them to obtain the sum of squares. The difference between 1 and the sum of squares is taken as the Gini coefficient of the word to be classified.

[0105] Taking the k-classification model as an example, the Gini coefficient of each word to be classified is calculated. Taking the i-th word to be classified as an example, its probability of appearing in the j-th category is p. j Then the Gini coefficient of the word to be classified is:

[0106]

[0107] Among them, G i Let be the Gini coefficient of the i-th word to be classified.

[0108] E3. The average Gini coefficient of each word to be classified is used as the score corresponding to the candidate sample.

[0109] For example, this embodiment provides a scoring method:

[0110]

[0111] Where gini_score is the score of the candidate sample, and p is the total number of words to be classified.

[0112] The scoring method provided in this application illustrates the role of each word in classification from a probabilistic perspective. If the same word appears in different samples, it indicates that the word has low information content and lacks the ability to distinguish between different samples. Conversely, a word that appears more frequently in a particular type of sample contains more information. Therefore, by representing the scores using the Gini coefficient, samples containing more information-rich words can be selected, resulting in a more balanced training sample. This scoring method can achieve balanced scoring based on word / character samples.

[0113] For each candidate sample, steps S203-S204 are used to process it and determine its corresponding target score.

[0114] S203. For each scoring method, determine the scores of all candidate samples corresponding to the scoring method, and record them as the scores to be processed. Based on each score to be processed, normalize the scores to be processed corresponding to the candidate samples to obtain the candidate scores corresponding to the scoring methods.

[0115] In this embodiment, the score to be processed can be understood as the score that needs to be processed; the candidate score can be understood as the score obtained after data processing.

[0116] In this embodiment, when normalizing scores, the scores determined using the same scoring method are normalized. For each scoring method, the scores of all candidate samples corresponding to that method are determined and recorded as unprocessed scores. Based on these unprocessed scores, the unprocessed scores corresponding to the candidate samples are normalized to a range, resulting in candidate scores for the candidate samples under that scoring method. For example, the sizes of the unprocessed scores are compared, and the largest and smallest scores are selected. Normalization is then performed based on the largest and smallest scores. This process is repeated sequentially, using the scores determined by each scoring method as unprocessed scores, until all scores are normalized into candidate scores.

[0117] As an optional embodiment, this optional embodiment normalizes the candidate scores corresponding to the candidate samples based on each candidate score to obtain the candidate scores corresponding to the candidate samples under the scoring method, which is optimized as follows:

[0118] F1. Sort the scores to be processed according to their size, determine the maximum value according to the first preset quantile, and determine the minimum value according to the second preset quantile.

[0119] In this embodiment, the first preset quantile and the second preset quantile can be preset, with the first preset quantile being less than 1 and the second preset quantile being greater than 0. For example, the first preset quantile is 85% and the second preset quantile is 15%. Since there may be outliers in the maximum and minimum values ​​of the data, this embodiment uses the first preset quantile and the second preset quantile as alternatives, which can make the score distribution more stable. That is, the maximum value determined by the first preset quantile is not the maximum value among the scores to be processed, and the minimum value determined by the second preset quantile is not the minimum value among the scores to be processed.

[0120] First and second preset quantiles are pre-set. The scores to be processed are sorted by size. The score corresponding to the first preset quantile in the ordered list is determined and taken as the maximum value. The score corresponding to the second preset quantile in the ordered list is determined and taken as the minimum value. For example, if the first preset quantile is 85% and the second preset quantile is 15%, the scores to be processed are sorted in ascending order. The score at the 85th percentile is selected as the maximum value, and the score at the 15th percentile is selected as the minimum value.

[0121] F2. Based on the maximum and minimum values, normalize the scores of the candidate samples to be processed to obtain the candidate scores of the candidate samples under the scoring method.

[0122] The scores to be processed are normalized to the interval formed by the maximum and minimum values ​​to obtain the candidate scores corresponding to the candidate samples under this scoring method.

[0123] For example, this application provides a normalized calculation formula:

[0124]

[0125] Among them, score i ' represents the candidate score of the i-th candidate sample. i The score is the unprocessed score for the i-th candidate sample. Q0.85 The maximum value, score Q0.15 It is the minimum value.

[0126] In this embodiment of the application, when normalizing the scores of candidate samples, it is necessary to process the scores of all candidate samples corresponding to each scoring method. Therefore, the maximum and minimum values ​​corresponding to each scoring method can be determined only once. After determining the maximum and minimum values, the scores of all candidate samples (i.e., the scores to be processed) corresponding to this scoring method are normalized.

[0127] As an optional embodiment, this optional embodiment is further optimized by: before normalizing the scores corresponding to the candidate samples based on each score to be processed, if the distribution of each score to be processed is an exponential distribution, performing a logarithmic transformation on each score to be processed.

[0128] Analyze the distribution of each score to be processed. If it follows an exponential distribution, apply a logarithmic transformation to make it approximate a linear or normal distribution. The base *n* of the logarithmic transformation can be determined based on the distribution of the original data. For example, analyze the number of data points in different intervals; if the interval [0,5] has 10 data points and the interval [5,10] has 100, then the base *n* of the logarithmic transformation would be 2. During the logarithmic transformation process, *n* can be adjusted based on the distribution before and after the transformation and specific business requirements until the transformed data approximates a linear or normal distribution.

[0129] For example, this application provides a formula for calculating logarithmic transformation:

[0130] Where "score" is the score after transformation, "score" is the score before transformation, and n is the base.

[0131] For example, embodiments of this application provide a score table, which includes scores obtained by scoring in four ways.

[0132] Table 4 Score Table

[0133] Candidate sample number emb_score iterm_score inter_score gini_score 1 5 50 0.8 0.2 2 15 20 0.6 0.5 3 3 80 0.7 0.3 … … … … …

[0134] In this embodiment of the application, when normalizing and logarithmically transforming the fractions, the same column of data in Table 4 is processed, that is, each column of data is normalized and logarithmically transformed separately.

[0135] S204. The target score of the candidate sample is obtained by weighting and summing all the candidate scores according to their respective weights.

[0136] In this embodiment, the weights corresponding to the candidate scores can be preset. For example, they can be preset based on specific data, training objectives, etc., or multiple sets of different weights can be provided in advance and stored in a data table. The weight values ​​in a set of weights can be equal or unequal. During use, the pre-generated data table is searched according to the specific data, training objectives, etc., a matching set of weights is selected, and the weighted sum of all candidate scores corresponding to the candidate sample is calculated based on this set of weights to obtain the target score of the candidate sample.

[0137] S205. Based on the target scores of the candidate samples, each candidate sample is screened to obtain at least one target sample.

[0138] The sample screening method provided in this invention offers multiple scoring methods for candidate samples, solving problems such as high screening costs and imbalance. Based on retrieval scores, weighting is applied through label consistency. Samples with highly similar content and consistent labels are assigned lower scores, while samples with highly similar content but different labels are assigned higher scores. Samples with highly similar content but different labels contain more information, thus screening out training samples that are distinct from those already in the training sample set, supplementing and enriching the training sample set, and improving the model's generalization ability. The method analyzes the impact of different characters / words on classification results using a regression model and calculates weight parameters. Target weight parameters with larger absolute values ​​are selected, and variance is calculated to determine the score of candidate samples. The variance represents the score of candidate samples; the larger the variance, the greater the discriminative power of each feature, and the more concentrated the reference features, thus screening out candidate samples containing more specific features. The method analyzes the role of each word in classification from a probabilistic perspective, screening out samples containing more information-rich words, making the training samples more comprehensive and balanced. Training the model based on the selected target samples can improve model iteration efficiency and prediction robustness.

[0139] Example 3

[0140] Figure 3 This is a schematic diagram of a sample screening device provided in Embodiment 3 of the present invention. Figure 3As shown, the device includes: a candidate sample set acquisition module 31, a scoring module 32, a target score determination module 33, and a target sample screening module 34.

[0141] The candidate sample set acquisition module 31 is used to acquire a candidate sample set, wherein the candidate sample set includes at least one candidate sample.

[0142] The scoring module 32 is used to score the classification generalization attribute of each candidate sample to determine the score corresponding to the candidate sample. The candidate sample has at least one score, and each score is scored in a different way. The scoring method includes the supplementary contribution of the candidate sample's sample content and classification label to the training sample set, as well as the distinguishing contribution of the word level classification in the candidate sample's sample content.

[0143] The target score determination module 33 is used to process the score corresponding to each candidate sample for each candidate sample and determine the target score of each candidate sample;

[0144] The target sample screening module 34 is used to screen each of the candidate samples based on the target score of the candidate samples to obtain at least one target sample.

[0145] The sample screening device provided in this invention solves the problems of high cost and imbalance in sample screening. It uses different scoring methods to score the classification generalization attributes of candidate samples, obtaining at least one score for each candidate sample. By processing multiple scores, a target score for the candidate sample is obtained, and then target samples are screened based on the target score. When scoring candidate samples, the contribution of the candidate sample's content and classification label to the training sample set is scored, effectively screening target samples used to enrich the training sample set and improving model generalization performance. The device also scores the contribution of words in the candidate sample content to different categories, screening samples containing more specific features based on the contribution of words to different categories. The sample screening method provided in this application can effectively screen samples, improving model iteration efficiency and prediction robustness.

[0146] Optionally, when the scoring method includes the supplementary contribution of the candidate sample's sample content and classification label to the training sample set, the scoring module 32 includes:

[0147] The matching score determination unit is used to retrieve training samples from a pre-generated training sample set based on the candidate samples, and determine the recall samples and the matching scores corresponding to the recall samples.

[0148] A weighting unit is used to weight each matching score based on the classification label of the recalled sample and the classification label of the candidate sample;

[0149] The first score determination unit is used to determine the score corresponding to the candidate sample based on each weighted matching score.

[0150] Optionally, the matching score determination unit is specifically used for: vectorizing the candidate samples to obtain candidate sample vectors; performing a search and matching based on the candidate sample vectors and the training sample vectors corresponding to the training samples to obtain a matching score for each training sample; and filtering each training sample based on the matching score to determine the recalled samples and the matching score corresponding to the recalled samples.

[0151] Optionally, the weighting unit is specifically used to: for each recalled sample, compare whether the classification label of the recalled sample is consistent with the classification label of the candidate sample; if they are consistent, determine that the weight of the matching score corresponding to the recalled sample is -1; if they are inconsistent, determine that the weight of the matching score corresponding to the recalled sample is 1.

[0152] Optionally, the score determination unit is specifically used for: summing each weighted matching score to obtain the score corresponding to the candidate sample; and / or calculating the ratio of each weighted matching score to the target length, summing all the obtained ratios to obtain the score of the candidate sample, wherein the target length is determined according to the length of the candidate sample.

[0153] Optionally, when the scoring method includes the distinguishing contribution of word-level classification in the sample content of the candidate sample, the scoring module 32 includes:

[0154] The perturbation sample collection unit is used to collect perturbation samples based on the candidate samples and the pre-trained classification model to obtain at least one perturbation sample.

[0155] The weight parameter determination unit is used to train the regression model based on the perturbation sample and obtain the weight parameters of the regression model.

[0156] The target weight parameter filtering unit is used to filter each weight parameter according to the absolute value of the weight parameters of the regression model to obtain a preset number of target weight parameters.

[0157] The second score determination unit is used to take the variance of each of the target weight parameters as the score corresponding to the candidate sample.

[0158] Optionally, when the scoring method includes the distinguishing contribution of word-level classification in the sample content of the candidate sample, the scoring module 32 includes:

[0159] The word segmentation unit is used to segment the candidate sample into words to obtain at least one word to be classified.

[0160] The Gini coefficient determination unit is used to determine the probability of each word to be classified in each category, calculate the sum of squares of each probability, and take the difference between 1 and the sum of squares as the Gini coefficient of the word to be classified.

[0161] The third score determination unit is used to take the average Gini coefficient of each of the words to be classified as the score corresponding to the candidate sample.

[0162] Optionally, the target score determination module 33 includes:

[0163] The normalization processing unit is used to determine the scores of all candidate samples corresponding to each scoring method, which are recorded as the scores to be processed, and to perform normalization processing on the scores to be processed corresponding to the candidate samples based on each score to be processed, so as to obtain the candidate scores corresponding to the candidate samples under the scoring method.

[0164] The target score determination unit is used to perform a weighted summation of all candidate scores corresponding to the candidate sample according to their respective weights to obtain the target score of the candidate sample.

[0165] Optionally, the normalization processing unit is specifically used to: sort the scores to be processed according to their size, determine the maximum value according to the first preset quantile, and determine the minimum value according to the second preset quantile; and perform normalization processing on the scores to be processed corresponding to the candidate samples based on the maximum value and the minimum value to obtain the candidate scores corresponding to the candidate samples under the scoring method.

[0166] Optionally, the target score determination module 33 also includes:

[0167] The logarithmic transformation unit is used to perform a logarithmic transformation on each of the candidate samples before normalizing the candidate samples based on each candidate sample's score, if the distribution of each candidate sample's score is an exponential distribution.

[0168] The sample screening device provided in the embodiments of the present invention can execute the sample screening method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0169] Example 4

[0170] Figure 4A schematic diagram of an electronic device 40 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0171] like Figure 4 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded into the RAM 43 from storage unit 48. The RAM 43 may also store various programs and data required for the operation of the electronic device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0172] Multiple components in electronic device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0173] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as sample screening methods.

[0174] In some embodiments, the sample screening method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the sample screening method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the sample screening method by any other suitable means (e.g., by means of firmware).

[0175] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0176] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0177] This invention provides a computer program product, which includes a computer program that, when executed by a processor, implements the sample screening method described in any embodiment of this invention.

[0178] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0179] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0180] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0181] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0182] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0183] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A sample screening method, characterized in that, include: Obtain a candidate sample set, wherein the candidate sample set includes at least one candidate sample; For each candidate sample, the classification generalization attribute of the candidate sample is scored to determine the score corresponding to the candidate sample. There is at least one score corresponding to the candidate sample, and each score is scored in a different way. The scoring method includes the supplementary contribution of the candidate sample's sample content and classification label to the training sample set, as well as the distinguishing contribution of the word level classification in the candidate sample's sample content. For each candidate sample, the score corresponding to the candidate sample is processed to determine the target score of the candidate sample; Based on the target scores of the candidate samples, each candidate sample is screened to obtain at least one target sample.

2. The method according to claim 1, characterized in that, When the scoring method includes the contribution of the candidate sample's content and classification label to the training sample set, the classification generalization attribute of the candidate sample is scored to determine the score corresponding to the candidate sample, including: Based on the candidate samples, the training samples in the pre-generated training sample set are retrieved to determine the recalled samples and the matching scores corresponding to the recalled samples; Each matching score is weighted based on the classification labels of the recalled samples and the classification labels of the candidate samples; The score corresponding to the candidate sample is determined based on each weighted matching score.

3. The method according to claim 2, characterized in that, The step of retrieving training samples from a pre-generated training sample set based on the candidate samples to determine the recalled samples and the matching scores corresponding to the recalled samples includes: The candidate samples are vectorized to obtain candidate sample vectors; The matching is performed based on the candidate sample vector and the training sample vector corresponding to the training sample to obtain the matching score for each training sample. The training samples are filtered based on the matching scores to determine the recalled samples and the matching scores corresponding to the recalled samples.

4. The method according to claim 2, characterized in that, The step of weighting each matching score based on the classification labels of the recalled samples and the classification labels of the candidate samples includes: For each recalled sample, compare whether the classification label of the recalled sample is consistent with the classification label of the candidate sample. If they are consistent, determine that the weight of the matching score corresponding to the recalled sample is -1; if they are inconsistent, determine that the weight of the matching score corresponding to the recalled sample is 1.

5. The method according to claim 2, characterized in that, Determining the score corresponding to the candidate sample based on each weighted matching score includes: The weighted matching scores are summed to obtain the score corresponding to the candidate sample; and / or, Calculate the ratio of each weighted matching score to the target length, sum all the ratios to obtain the score of the candidate sample, wherein the target length is determined based on the length of the candidate sample.

6. The method according to claim 1, characterized in that, When the scoring method includes the distinguishing contribution of word-level classification in the sample content of the candidate sample, the classification generalization attribute of the candidate sample is scored to determine the score corresponding to the candidate sample, including: Based on the candidate samples and the pre-trained classification model, perturbation samples are collected to obtain at least one perturbation sample; The regression model is trained based on the perturbation samples to obtain the weight parameters of the regression model; The weight parameters are filtered according to the absolute value of the weight parameters of the regression model to obtain a preset number of target weight parameters. The variance of each of the target weight parameters is used as the score corresponding to the candidate sample.

7. The method according to claim 1, characterized in that, When the scoring method includes the distinguishing contribution of word-level classification in the sample content of the candidate sample, the classification generalization attribute of the candidate sample is scored to determine the score corresponding to the candidate sample, including: The candidate samples are segmented to obtain at least one word to be classified; For each word to be classified, determine the probability of the word in each category, calculate the sum of squares of each probability, and use the difference between 1 and the sum of squares as the Gini coefficient of the word to be classified. The average Gini coefficient of each of the words to be classified is used as the score corresponding to the candidate sample.

8. The method according to claim 1, characterized in that, The step of processing the scores corresponding to the candidate samples to determine the target scores of the candidate samples includes: For each scoring method, the scores of all candidate samples corresponding to the scoring method are determined and recorded as the scores to be processed. Based on each score to be processed, the scores to be processed corresponding to the candidate samples are normalized to obtain the candidate scores corresponding to the candidate samples under the scoring method. The target score of the candidate sample is obtained by weighting and summing all the candidate scores corresponding to the candidate sample according to their respective weights.

9. The method according to claim 8, characterized in that, The step of normalizing the candidate scores corresponding to the candidate samples based on each of the candidate scores to be processed, to obtain the candidate scores corresponding to the candidate samples under the scoring method, includes: The scores to be processed are sorted according to their size, the maximum value is determined according to the first preset quantile, and the minimum value is determined according to the second preset quantile. Based on the maximum and minimum values, the scores to be processed corresponding to the candidate samples are normalized to obtain the candidate scores corresponding to the candidate samples under the scoring method.

10. The method according to claim 8, characterized in that, Before normalizing the scores corresponding to the candidate samples based on each of the scores to be processed, the method further includes: If the distribution graph of each of the scores to be processed is an exponential distribution, then a logarithmic transformation is performed on each of the scores to be processed.

11. An electronic device, characterized in that, The electronic device includes: At least one processor, and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the sample screening method according to any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the sample screening method according to any one of claims 1-10.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the sample screening method according to any one of claims 1-10.

Images