An unsupervised core set selection method and device for context learning

Abstract

The present application belongs to the technical field of unsupervised machine learning, and more particularly relates to an unsupervised core set selection method and device for context learning. The method acquires an unlabeled data set and a labeling budget, constructs a joint optimization diversity and representation sub-module objective function; designs a normalized diversity evaluation function based on a determinant point process, and designs a process-aware representation evaluation function based on local density and redundancy penalty; dynamically adjusts the weight through an adaptive balance factor, iteratively selects samples using a greedy algorithm, and outputs a small-scale high-quality core set. The present application solves the technical problems of the existing unsupervised core set selection method, such as local redundancy, rigid balance between diversity and representation, large computational overhead, dependence on downstream models and labeling information, and inability to adapt to large model context learning high-quality core set selection in low-labeling budget scenarios.

Description

Technical Field

[0001] This invention belongs to the technical field of unsupervised machine learning, and more specifically, relates to an unsupervised kernel set selection method and apparatus for context learning. Background Technology

[0002] Unsupervised core set selection involves choosing a small subset from a large-scale unlabeled dataset that effectively represents the distribution of the original data. This selected core set is then manually labeled and applied to downstream tasks. Taking context learning tasks as an example, the performance of a large model is highly dependent on the quality of the examples: selecting representative and diverse examples can activate the model's knowledge and reasoning abilities, while low-quality examples may introduce bias or noise. When the budget for manual labeling is limited (e.g., a few dozen samples), the core set selected by this technique can significantly reduce data labeling costs and computational resource overhead, addressing a critical problem in resource-constrained machine learning scenarios. Existing unsupervised core set selection methods can be broadly categorized into model-independent methods and model-based methods.

[0003] Chinese patent document CN114298304A discloses an active learning method, apparatus, electronic device, and readable storage medium, belonging to the model-independent method category. It proposes clustering the entire dataset and then selecting samples closer to each cluster center to ensure diversity and representativeness. Secondly, Chinese patent document CN114169470A discloses an artificial intelligence learning method based on a target model and dual sampling of samples, belonging to the model-based method category. It selects samples that generate high confidence in the model's output for labeling and iteratively trains the model and selects samples.

[0004] In summary, existing model-agnostic methods generally suffer from two limitations: First, they lack neighborhood awareness of the selection process, leading to local redundancy in the selected core set and repeated sampling from high-density regions. Second, they lack an adaptive balancing mechanism, failing to flexibly adjust the priority between diversity and representativeness based on different labeling budgets or dynamic selection processes. Furthermore, model-based methods suffer from significant computational bottlenecks, requiring model inference or iterative training across the entire candidate data pool to estimate sample importance. They are heavily reliant on specific downstream model architectures, making them difficult to transfer across different models and exhibiting poor scalability. Additionally, using training losses or gradients as the basis for sample selection requires substantial label information, making them unsuitable for scenarios with low labeling budgets. Summary of the Invention

[0005] The present invention aims to overcome at least one of the defects of the prior art and provide an unsupervised core set selection method for context learning. This method addresses the technical problems of existing unsupervised core set selection methods, such as local redundancy, rigid balance between diversity and representativeness, high computational cost, dependence on downstream models and annotation information, and inability to adapt to high-quality core set selection for large model context learning in scenarios with low annotation budgets.

[0006] This method jointly optimizes the diversity and representativeness of selected samples through a monotonic submodulus objective function. Regarding diversity, a diversity evaluation method based on determinant-based point processes is proposed; regarding representativeness, a process-aware dynamic representativeness evaluation method is designed, employing a process decay factor to adjust the global density score, ensuring priority selection of samples from high-density regions, while effectively avoiding local overlap by penalizing candidate points within the neighborhood radius of selected samples. To adapt to robustness requirements under different labeling budgets, this invention introduces an adaptive balancing mechanism to balance the weights between diversity and representativeness. Compared to existing technologies, this invention requires no model iterative training or manual intervention, can be directly applied to completely unlabeled data pools, and the efficient greedy solution algorithm employed has a theoretically provable optimal approximation, providing a robust and effective core set selection scheme for few-shot learning.

[0007] The detailed technical solution of this invention is as follows: An unsupervised core set selection method for context learning, the method comprising: S1. Obtain the unlabeled text dataset. With budget That is, the number of core set samples, which is used to extract text datasets using a feature extractor. Candidate samples in the dataset are converted into vector representations; S2. Construct a sub-modulus objective function that jointly optimizes sample diversity and representativeness: (1) (2) In formula (1)-(2), For the core set to be selected; For the total score function; This is a normalized submodulus diversity evaluation function based on determinant point processes. For the representative evaluation function of the local density sub-modules of process awareness, An adaptive balance factor between diversity and representativeness; S3, Initialize candidate core sets For an empty set, a greedy algorithm iteratively selects the candidate sample with the highest sub-modulus objective function score for each joint optimization of sample diversity and representativeness, and adds it to the candidate core set. Until the scale reaches the marked budget ; S4. Output the final core set after the iteration is completed. And then it was handed over to experts for annotation.

[0008] Furthermore, the construction of the sub-model objective function that jointly optimizes sample diversity and representativeness specifically includes: S21. Normalized Submodulus Diversity Evaluation Function Based on Determinant-Based Point Processes ; Unlike previous works that assessed diversity based on DPP, this method introduces a normalization factor to limit the diversity value range to a specific range, in order to prevent the value range from dominating the objective function due to an excessively large range.

[0009] Specifically, this method introduces a theoretical upper limit for DPP, which is determined by the entire dataset. The largest eigenvalue of the kernel matrix The derivation yields that the proposed normalized sub-modulus diversity term is: (3) In formula (3), It is a matrix, whose first... i Line 1 j The column elements are selected samples, that is, samples that have been added to the core set of candidates. kernel similarity, For the i-th selected sample, For the first j For each selected sample, the kernel similarity is calculated using the inner product of the sample vectors. It is a dataset Kernel similarity matrix between candidate samples; This is a custom small positive offset, used to ensure numerical stability. Represents the identity matrix. For the entire dataset The largest eigenvalue of the kernel matrix; The numerator of formula (3) is the addition of a small offset. The logarithmic determinant terms are used to ensure numerical stability. This method takes... To ensure The denominator is the theoretical bound of the DPP, used for normalization to prevent unboundedness. In the objective function China holds a dominant position.

[0010] This function is a submodular function, meaning that adding elements to a smaller set yields a larger marginal gain, while a more diverse set typically corresponds to a larger one. Fraction.

[0011] S22, Representative evaluation function of local density sub-modes for process awareness; To formalize the concepts of density and local redundancy, this method defines a neighborhood range, i.e., a distance radius. Based on this, firstly, density score is defined. Measuring the distance to a candidate sample The larger the number of other samples within the candidate sample, the more representative the candidate sample is; secondly, a redundancy decay factor is defined. Penalize cases that are too close to the selected sample.

[0012] Combination density score and redundancy decay factor Build a unified : (4) Inspired by robust statistical methods, this method uses the sample to its first... The median of nearest neighbor distance ( To estimate the radius Compared to the mean, the median is less sensitive to outliers and long-tailed distributions. Specifically, to ensure that each selected sample in the core set covers the effective portion of the data, this method will... With budget Perform reverse scaling: smaller Each selected sample needs to cover more data. The corresponding increase is necessary. Therefore, the settings are... Candidate samples are One. Among them This is a scaling factor used to determine the neighborhood size parameter. The value of . Let For dataset The Middle i Sample Embedded vector, For its first The nearest neighbor embedding vector: (5) based on ,index Measure the selected sample The number of unlabeled samples in the surrounding area, while Calculated from the current core candidate set; (6) (7).

[0013] In formulas (6)-(7), express Any sample in the middle, express The sample with the highest density (the one with the most surrounding unlabeled samples). These are their embedding vectors; This represents the other selected samples in the current candidate core set S. The embedding vector; This is an indicator function.

[0014] This formula embodies an information theory intuition: under the assumption that the occurrence of samples in a local neighborhood follows a Poisson-like process, the marginal information provided by new samples will decay exponentially as the number of samples in the neighborhood increases.

[0015] Furthermore, the adaptive estimation of the balance factor specifically includes: A key component of this invention is the balance factor. It is used to control the trade-off between diversity and representativeness. This invention proposes a method based on two key considerations: amplitude scaling and budget awareness. Adaptive estimation is performed. Specifically, this invention will... Defined as: (8) In formula (8), It is a budget-related item; In the second term, the expected ratio of the numerator to the denominator is used as an amplitude scaling term to balance the diversity gain. With representative gain The typical amplitude. It should be noted that, after estimation, Set a fixed positive value in the core set selection to ensure the objective function... Sub-model properties.

[0016] Preferably, the budget adaptive item Specifically: In tight budget situations, maximizing representativeness is crucial for capturing the core data distribution. As budgets increase, diversity becomes even more important to avoid redundancy. To automate this prioritization, this invention designs a budget-aware item. Its value depends on the current budget. With dataset size The ratio, and through a constant C Define the compensation coefficient.

[0017] (9) This formula guarantees This allows for a smooth and monotonous transition within the budget. That is, for smaller budgets, >$1 to enhance representativeness, while for larger budgets, It tends to favor diversity.

[0018] Preferably, the amplitude scaling term Specifically: To balance diversity gains With representative gain The scaling factor is estimated by using the typical amplitude of the diversity gain to the typical amplitude of the representative gain, which is obtained by performing a greedy probe once.

[0019] Specifically, without introducing a balance factor (or equivalent) Under the condition of ), according to the optimization objective of formula (1), execute One greedy step to satisfy This value is guaranteed Greater than the stated budget To enhance the stability of the estimation, while being much smaller To focus on active samples and avoid long-tailed noise. Through empirical optimization, the following settings were adopted: Record each greedy step. Diversity gain and representative gain Calculate the expected ratio of the two as the amplitude ratio. ; At every step In the middle, let diversity gain be set , indicating the first Step-by-step selected sample set No. Step-by-step selected sample set diversity function difference; Similarly, representative gain Defined as .

[0020] This method is based on the unweighted sum of marginal gains. implement Step-by-step greedy choice 。 for The empirical average of the steps. The reasonable range of values ​​for is To ensure that the probe focuses on the active samples in the actual selection, based on experimental experience, the preferred setting is... Because it is longer than the budget The sequence can provide a stable average gain estimate while the computational cost remains low.

[0021] Furthermore, S3 specifically includes: S3.1 Initialize Selected Sets It is an empty set; S3.2 proceed The greedy selection process involves executing the constructed joint optimization objective function for sample diversity and representativeness to obtain the total score for each candidate sample. The sample with the highest total score in each iteration is then added to the core candidate set. ; In the During the greedy process, From currently unselected candidate samples Among them, select the sample with the largest marginal gain: (10) (11) in, For the front The selected core set after the first step For the current number The optimal sample selected in the first step. For the first The core set to be selected after the first step.

[0022] Repeat the above process until Get the final core set The core set obtained by this greedy algorithm Compared to the theoretically optimal set It can reach no more than The error is approximately guaranteed.

[0023] In another aspect of the invention, an electronic device is also provided, comprising: At least one processor; and A memory that stores instructions that, when executed by the at least one processor, cause the at least one processor to perform an unsupervised core set selection method for context learning as described above.

[0024] In another aspect of the invention, a computer-readable storage medium is also provided, which stores executable instructions that, when executed, cause the machine to perform an unsupervised kernel set selection method for context learning as described above.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention provides an unsupervised core set selection method and apparatus for context learning, which proposes a process-aware diversity and representativeness balance mechanism. This mechanism can solve the problem that traditional methods cannot actively suppress local redundancy in terms of geometric features, leading to information overlap within the core set. Specifically, it introduces a function based on neighborhood local density and an exponentially decaying redundancy penalty term in the representativeness evaluation: the former ensures that samples located in high-density areas of the data distribution are selected first to cover the main data; the latter applies a marginal penalty to candidate samples that are too geometrically close to the selected samples. Compared with the traditional method of statically balancing diversity and representativeness in the objective function using fixed weights, the present invention can actively suppress local redundancy from the underlying geometric structure of the data distribution, effectively avoiding information overlap within the core set.

[0026] (2) The present invention provides an unsupervised core set selection method and apparatus for context learning, proposing a global sub-module objective function that jointly optimizes diversity and representativeness, supports efficient greedy search, and obtains at least 1 The theoretical approximation guarantee of the optimal solution at 1 / e combines strict error controllability with high computational efficiency. In this invention, the ranges of the diversity and representativeness evaluation functions are normalized to the same scale interval, and the typical magnitude difference between the marginal gain of diversity and the marginal gain of representativeness is estimated online using probes, achieving adaptive estimation of the balance factor. Compared to static weighting methods that rely on human experience or cross-validation, this invention can adapt to datasets of different sizes and feature distributions, avoiding repeated parameter tuning.

[0027] (3) The present invention provides an unsupervised core set selection method and apparatus for context learning, proposing a model-independent, unsupervised core set selection method. The selection process is completely independent of the downstream task model structure, requiring no model iterative training or manual intervention. Compared with model-based methods (such as those relying on confidence sampling or reinforcement learning), the present invention does not require model inference or iterative training, avoiding computational bottlenecks. At the same time, since it does not rely on labeled signals, the present invention remains effective even with low label budgets, and can quickly output high-quality core sets during the data preprocessing stage, directly serving any subsequent task (such as context learning, active learning, etc.), and possessing good task generalization and deployment efficiency. Attached Figure Description

[0028] Figure 1 This is a flowchart of an unsupervised core set selection method for context learning as described in this invention.

[0029] Figure 2 This is a schematic diagram of the synergistic selection of diversity and representativeness in Embodiment 1 of the present invention, wherein (a) is a schematic diagram of preferred diversity under the same representativeness; and (b) is a schematic diagram of preferred representativeness under the same diversity.

[0030] Figure 3 These are distribution diagrams of the core sets selected by each method in Embodiment 1 of the present invention in the embedding space, wherein (a) is the distribution diagram of the core sets selected by the comparison method IDEAL in the embedding space; (b) is the distribution diagram of the core sets selected by the comparison method RDSS in the embedding space; (c) is the distribution diagram of the core sets selected by the comparison method InfoMax in the embedding space; and (d) is the distribution diagram of the core sets selected by the method of the present invention in the embedding space.

[0031] Figure 4 This is a dynamic balance diagram of diversity and representativeness in Embodiment 1 of the present invention, wherein (a) is a dynamic balance diagram of diversity and representativeness when the budget m=18; and (b) is a dynamic balance diagram of diversity and representativeness when the budget m=100. Detailed Implementation

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0033] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0034] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0035] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0036] Example 1 Explanation of technical terms used in this embodiment: Contextual learning is a reasoning paradigm for large models, in which the model understands the task intent and generates output through a small number of examples provided in the input during the reasoning phase, without any updates or fine-tuning of the model parameters.

[0037] Core set selection: Core set selection is a data subset selection technique that aims to select a subset from a large dataset that is much smaller than the original set, so that the performance of downstream tasks performed using the subset is as close as possible to the performance obtained on the full dataset.

[0038] Unsupervised core set selection: a technique for core set selection that relies only on the features of the samples themselves in the complete absence of label information.

[0039] Submodular Function: An important property of set functions, referring to diminishing marginal benefit: the gain obtained by adding an element to a smaller set is greater than or equal to the gain obtained by adding the same element to a larger set. Formally, for a finite set the set function on , if for any set , and satisfies , then it is called a submodular function, and this property characterizes diminishing marginal benefit.

[0040] Determinantal Point Process (DPP): A probabilistic model for selecting a highly diverse subset from a set of objects. DPP measures the quality of a subset through the determinant of a positive definite kernel matrix, where a larger determinant indicates lower similarity between elements in the subset, that is, higher diversity.

[0041] reference Figure 1 , the present embodiment provides an unsupervised core set selection method for in-context learning, and the method comprises: Step S1: Data preparation and vector conversion; Acquire an unlabeled text dataset and a labeling budget that is, the number of core set samples, and use a feature extractor to convert the candidate samples in the text dataset into vector representations.

[0042] Specifically, the step S1 specifically comprises: S1.1 Given an unlabeled text dataset .

[0043] Given an unlabeled text dataset , which contains candidate samples, a labeling budget is given , usually , for example .

[0044] The goal of unsupervised core set selection is to select a subset of size from as the core set, which is used for expert labeling and forms an effective prompt for in-context learning.

[0045] ​S1.2 uses a feature extractor to extract each candidate sample The feature extractor can be converted into a vector representation. It can employ any pre-trained sentence encoding model, preferably Sentence-BERT or SimSCE.

[0046] Step S2: Construct a sub-model objective function that jointly optimizes sample diversity and representativeness: (1) (2) Formula (1)-(2), For the core set to be selected; For the total score function; This is a normalized submodulus diversity evaluation function based on determinant point processes. For the representative evaluation function of the local density sub-modules of process awareness, This invention provides an adaptive balance factor between diversity and representativeness. and Designed as a submodular function, i.e., a set function with diminishing marginal returns. Based on the closure property of submodular functions, their nonnegative linear combination... It is also a submodular function, so it can be solved efficiently using a greedy algorithm and the standard approximation guarantee can be obtained.

[0047] This invention encourages diverse choices based on determinant point processes (DPP). According to DPP, subsets... The diversity of its matrix and the determinant of its positive semidefinite kernel matrix Proportional, where the kernel matrix The element Calculate samples and The kernel similarity between the sets can use any positive semi-definite kernel, such as the RBF kernel or the inner product kernel. diversity It can be measured by this determinant.

[0048] Furthermore, the construction of the sub-model objective function that jointly optimizes sample diversity and representativeness specifically includes: S21. Normalized Submodulus Diversity Evaluation Function Based on Determinant-Based Point Processes ; Unlike previous works that assessed diversity based on DPP, this method introduces a normalization factor to limit the diversity value range to a specific range, in order to prevent the value range from dominating the objective function due to an excessively large range.

[0049] Specifically, this method introduces a theoretical upper limit for DPP, which is determined by the entire dataset. The largest eigenvalue of the kernel matrix The derivation yields that the proposed normalized sub-modulus diversity term for: (3) In formula (3), It is a matrix, whose first... i Line 1 j The column elements are selected samples, that is, samples that have been added to the core set of candidates. kernel similarity, For the i-th selected sample, For the first j For each selected sample, this embodiment uses the inner product of the sample vectors to calculate the kernel similarity. It is a dataset The kernel similarity matrix between candidate samples (which can be represented in the form of Gaussian kernel, polynomial kernel, etc.); This is a custom small positive offset, used to ensure numerical stability. Represents the identity matrix. For the entire dataset The largest eigenvalue of the kernel matrix; The numerator of formula (3) is the addition of a small offset. The logarithmic determinant terms are used to ensure numerical stability. This method takes... To ensure The denominator is the theoretical bound of the DPP, used for normalization to prevent unboundedness. In the objective function China holds a dominant position.

[0050] This function is a submodular function, meaning that adding elements to a smaller set yields a larger marginal gain, while a more diverse set typically corresponds to a larger one. Fraction.

[0051] Preferably, the present invention provides a normalized sub-modulus diversity evaluation function. The proofs of the submodularity and range are as follows: The The range of values ​​is The proof is as follows: Let... for The eigenvalues ​​of . According to fundamental linear algebra: (12) (13) Therefore, there is All eigenvalues ​​satisfy ,and : (14) Therefore, there is In summary

[0052] The It is a sub-module function. Because yes The positive linear transformation, and it is known Since the function is a submodular function on a positive definite matrix, the marginal gain inequality still holds. This is a sub-module function.

[0053] S22, Representative Evaluation Function of Local Density Submodules for Process Awareness — ; This invention proposes a representative submodule evaluation function, the purpose of which is to ensure that the selected core set can effectively represent the entire unlabeled data pool. The design philosophy of this function is that... Each sample in the dataset should be "covered" or well approximated by at least one sample in the selected core set, thus preserving key features of the data distribution and providing informative examples for contextual learning. This idea is consistent with existing approaches to selecting prototypes to generalize datasets in data subset selection and active learning. To achieve the above idea, this invention evaluates the representativeness of candidate samples from two complementary perspectives: (1) Density: The selected samples should be located in the unlabeled data pool. (2) Local redundancy: The selected samples should be geometrically far away from the already selected set. The samples in the core set are selected to prevent information overlap and to encourage diversity within the core set.

[0054] To formalize the concepts of density and local redundancy, this method first defines a neighborhood range, i.e., a distance radius.

[0055] Based on this, firstly, density score is defined. Measuring the distance to a candidate sample The larger the number of other samples within the candidate sample, the more representative the candidate sample is; secondly, a redundancy decay factor is defined. Penalize cases that are too close to the selected sample.

[0056] Combination density score and redundancy decay factor Build a unified : (4) Inspired by robust statistical methods, this method uses the sample to its first... The median of nearest neighbor distance ( To estimate the radius Compared to the mean, the median is less sensitive to outliers and long-tailed distributions. Specifically, to ensure that each selected sample in the core set covers the effective portion of the data, this method will... With budget Perform reverse scaling: smaller Each selected sample needs to cover more data. The corresponding increase is necessary. Therefore, the settings are... Candidate samples are One. Among them This is a scaling factor used to determine the neighborhood size parameter. The value of . Based on Empirical studies on density estimation show that Preferred settings Within a certain range, it can ensure a suitable neighborhood and stable local statistics while avoiding excessive bias. The definition of the neighborhood and the local density estimation can automatically adapt to the inherent distribution of the data and the selected scale parameter. ,and It can be associated with the available annotation budget. Let For dataset The Middle i Sample Embedded vector For its first The nearest neighbor embedding vector: (5) based on ,index Measure the selected sample The number of unlabeled samples in the surrounding area, while Calculated from the current core candidate set; (6) (7).

[0057] In formulas (6)-(7), express Any sample in the middle, express The sample with the highest density (the one with the most surrounding unlabeled samples). These are their embedding vectors; This represents the other selected samples in the current candidate core set S. These represent the selected samples respectively. The embedding vector; This is an indicator function.

[0058] This formula embodies an information theory intuition: under the assumption that the occurrence of samples in a local neighborhood follows a Poisson-like process, the marginal information provided by new samples will decay exponentially as the number of samples in the neighborhood increases.

[0059] Preferably, the present invention provides a local density submode representativeness evaluation function. The proofs of the submodularity and range are as follows: The range of values ​​is Because for each selected sample ,have: (15) therefore, With normalization They are comparable.

[0060] It is a submodular function, and the factor modulo property requires that for any set and any sample Marginal gain satisfy Considering Add to collection Marginal gain: (16) because ,therefore, .Add to The marginal gain decreases as the set increases, which satisfies the property of diminishing marginal returns. This is a sub-module function.

[0061] Furthermore, the adaptive estimation of the balance factor specifically includes: A key component of this invention is the balance factor. It is used to control the trade-off between diversity and representativeness. This invention proposes a method based on two key considerations: amplitude scaling and budget awareness. Adaptive estimation is performed. Specifically, this invention will... Defined as: (8) In formula (8), It is a budget-related item; In the second term, the expected ratio of the numerator to the denominator is used as an amplitude scaling term to balance the diversity gain. With representative gain The typical amplitude. It should be noted that, after estimation, Set a fixed positive value in the core set selection to ensure the objective function... Sub-model properties.

[0062] Preferably, the budget adaptive item Specifically: In tight budget situations, maximizing representativeness is crucial for capturing the core data distribution. As budgets increase, diversity becomes even more important to avoid redundancy. To automate this prioritization, this invention designs a budget-aware item. Its value depends on the current budget. With dataset size The ratio, and through a constant C Define the compensation coefficient.

[0063] (9) This formula guarantees This allows for a smooth and monotonous transition within the budget. That is, for smaller budgets, >1 to enhance representativeness, while for larger budgets, It tends to favor diversity.

[0064] Preferably, the amplitude scaling term Specifically: To balance diversity gains With representative gain The scaling factor is estimated by using the typical amplitude of the diversity gain to the typical amplitude of the representative gain, which is obtained by performing a greedy probe once.

[0065] Specifically, without introducing a balance factor (or equivalent) Under the condition of ), according to the optimization objective of formula (1), execute One greedy step to satisfy This value is guaranteed Greater than the stated budget To enhance the stability of the estimation, while being much smaller To focus on active samples and avoid long-tailed noise. Through empirical optimization, the following settings were adopted: Record each greedy step. Diversity gain and representative gain Calculate the expected ratio of the two as the amplitude ratio. ; At every step In the middle, let diversity gain be set , The marginal gain of diversity represents the first... Step-by-step selected sample set No. Step-by-step selected sample set diversity function difference; Similarly, at each step In the middle, let the representative gain be... , Let be the representative marginal gain, representing the th Step-by-step selected sample set No. Step-by-step selected sample set Representative function difference.

[0066] In the core set selection method proposed in this invention, how to coordinate diversity and representativeness in sample selection, such as... Figure 2 As shown in the figure, the process of selecting the first initial sample is described. Then, select the next sample. Conceptual trade-offs regarding time: Figure 2 (a) in the figure is a schematic diagram of the preference for diversity under the same representativeness, which shows that when the representativeness of different candidate samples is similar (the local density is similar), the sample with more diversity (farther from the selected sample) is preferred, that is, it is both representative and more diverse. Figure 2 (b) in the figure is a schematic diagram of the preferred representative under the same diversity, which shows that when the diversity of different candidate samples is comparable (the distance from the selected sample is comparable), the more representative sample (the one with greater local density) is selected first, that is, the sample with both diversity and greater representativeness is selected first. In practice, any sample The selections are all based on their relative to the currently selected sample set. The marginal gain (rather than just relative to the last sample) is considered. This process-aware evaluation method enables the present invention to effectively avoid local redundancy and over-marginalization in the greedy selection process.

[0067] This method is based on the unweighted sum of marginal gains. implement Greedy choice 。 for The empirical average of the steps. The reasonable range of values ​​for is To ensure that the probe focuses on the active samples in the actual selection, based on experimental experience, the preferred setting is... Because it is longer than the budget The sequence can provide a stable average gain estimate while the computational cost remains low.

[0068] S3. Iterate using a greedy algorithm until the size of the core set reaches the annotation budget; Initialize selected set Given an empty set, a greedy algorithm iteratively selects the candidate sample with the highest total score in each iteration until the core set is reached. The scale reached the standard budget ; Furthermore, S3 specifically includes: S3.1 Initialize Selected Sets It is an empty set. ; S3.2 proceed The greedy selection process involves executing a jointly optimized submodal objective function that balances sample diversity and representativeness to obtain the total score for each candidate sample. The sample with the highest total score in each iteration is then added to the core candidate set. ; In the Step-by-step greedy process ( When ), select from the currently unselected candidate samples. Among them, select the sample with the largest marginal gain: (10) (11) in, For the front The selected core set after the first step For the current number The optimal sample selected in the first step. For the first The core set to be selected after the first step.

[0069] Repeat the above process until To obtain the final core set The core set obtained by this greedy algorithm Compared to the theoretically optimal set It can reach no more than The error approximation is guaranteed, that is... f ( ).

[0070] Through the process-aware selection mechanism described above, this invention can dynamically balance diversity and representativeness under different annotation budgets to obtain a high-quality unsupervised core set.

[0071] S4. Output the final core set after the iteration is completed. And then it was handed over to experts for annotation.

[0072] S4.1 output includes The final core set of samples ; S4.2 will The annotations are then handed over to experts for use in downstream tasks (such as context learning).

[0073] To illustrate the effectiveness of the method proposed in this invention in selecting samples, this embodiment visualizes the distribution of the selected samples in the embedding space (as described in step S1.2, the distribution of the representation vectors of the samples after feature extraction). Using t-SNE dimensionality reduction, the distribution effect of the selected core set in the embedding space is shown below. Figure 3 As shown in the image, blue dots represent unlabeled samples, and red dots represent selected samples.

[0074] This embodiment compares with the following existing methods: IDEAL is a core set selection method based on nearest neighbor similarity, derived from the paper "IDEAL: influence-driven selective annotations empower in-context learners in large language models"; InfoMax is a core set selection method based on information entropy, originating from the paper "Data pruning by information maximization". RDSS is a core set selection method based on distribution approximation, derived from the paper "Enhancing semi-supervised learning via representative and diverse sample selection."

[0075] This embodiment uses the MRPC dataset as a typical example for illustration. The MRPC dataset comes from the paper "Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news source". Figure 3 (a), (b), (c), and (d) in the figure are respectively the distribution effect of the selected core sets in the embedding space of the comparison method IDEAL, the comparison method RDSS, the comparison method InfoMax, and the method of the present invention. The visualization results are as follows: For the comparison method, see IDEAL. Figure 3 (a) in the text, and see InfoMax for the comparison method. Figure 3In (c), the above tends to over-select samples from high-density regions, leading to local redundancy. In contrast, the method of this invention is described in [reference to [reference to another document]]. Figure 3 In (d), neighborhood redundancy is effectively suppressed, therefore this method introduces redundancy penalty within the neighborhood of the selected sample; For the comparison method RDSS, see [link to RDSS]. Figure 3 (b) in the text, and see the comparison method IDEAL. Figure 3 In (a) above, outliers or marginal samples are often selected, which have low representativeness. In contrast, the method of this invention is described in [reference needed]. Figure 3 In (d), edge samples are not over-selected because this method assigns a lower selection priority to edge samples and focuses more on more representative areas of the data distribution.

[0076] To demonstrate that this method can achieve a dynamic balance between process-aware diversity and representativeness, this embodiment visualizes the contribution ratios of diversity and representativeness in the selection process. Specifically, for each step t=1,…,m, the gain of the representativeness term is calculated. The proportion of total gain, this ratio quantifies the instantaneous balance between diversity and representativeness gain, such as Figure 4 As shown, the horizontal axis represents the selection step t, and the vertical axis represents the ratio. The ratio recorded by this method is... The contribution ratio is represented by the blue line; in contrast, this embodiment also shows two variants: When the balance factor α is removed The contribution ratio is shown by the red dashed line; secondly, the budget perception coefficient is removed. (m) The contribution ratio is shown by the green line.

[0077] This example demonstrates the visualization results on the MRPC dataset. It shows the results under two different annotation budgets: the dynamic balance between diversity and representativeness when the budget is smaller (m=18). Figure 4 As shown in (a) above; and the dynamic balance between diversity and representativeness when the budget is larger (m=100), as shown in (a). Figure 4 As shown in (b) of the diagram.

[0078] (1) This method balances representativeness and diversity. Without the balancing factor α (red line), representativeness dominates the selection process. This dominance stems from the mismatch in magnitudes of the Div(S) and Rep(S) functions, leading to a significant bias towards representativeness in the selection. However, by introducing scaling based on the magnitude ratio in α, the two terms become comparable in magnitude (green line), and their contribution ratio stabilizes around 0.5, indicating that diversity and representativeness exert similar influences on the selection.

[0079] (2) Budget perception coefficient (m) Balances as the budget changes. (Introduction) After (m), the balance will be further adjusted based on the marked budget m (blue line). For smaller budgets m=18, the representative contribution is slightly higher than 0.5, and more emphasis is placed on representativeness to ensure sufficient coverage when the budget is tight; for larger budgets m=100, the trajectory is slightly lower than 0.5, and the impact of diversity is increased to reduce redundancy.

[0080] (3) The selection process dynamically adapts to the selected samples. The contribution trajectory (blue line) of this method exhibits two characteristics. First, it presents a distinct sawtooth pattern. Since both Rep(S) and Div(S) are monotonic submodular functions, if one of them is absolutely dominant, a smooth monotonic curve will be produced; the observed oscillations indicate that there is an active trade-off between the two during the selection process. Furthermore, representativeness Rep(S) contributes more in the early steps, while diversity Div(S) has a greater influence in the later stages. This behavior naturally stems from the design of the representativeness term—it dynamically considers the selected samples and penalizes local redundancy. As more neighbors within a radius r are selected, the marginal gain of representativeness gradually decays, thus allowing diversity to guide subsequent selections. This dynamic mechanism ensures that the core set achieves broad coverage with minimal redundancy.

[0081] Example 2 This embodiment provides an apparatus for an unsupervised core set selection method for context learning, the apparatus comprising: At least one processor; and A memory that stores instructions that, when executed by the at least one processor, cause the at least one processor to perform an unsupervised core set selection method for context learning as described above.

[0082] In this embodiment, the device can be an electronic device, including but not limited to: personal computer, server computer, workstation, desktop computer, laptop computer, notebook computer, mobile computing device, smartphone, tablet computer, cellular phone, personal digital assistant (PDA), handheld device, messaging device, wearable computing device, consumer electronic device, etc.

[0083] Example 3 This embodiment also provides a computer-readable storage medium storing executable instructions that, when executed, cause the machine to perform an unsupervised kernel set selection method for context learning as described above.

[0084] Specifically, a system or apparatus equipped with a readable storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system or apparatus can read and execute the instructions stored in the readable storage medium.

[0085] In this case, the program code read from the readable medium itself can perform the functions of any of the above embodiments, and therefore the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of this specification.

[0086] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0087] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0090] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0091] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. An unsupervised core set selection method for context learning, characterized in that, The method includes: S1. Obtain the unlabeled text dataset and the labeled budget, i.e., the core set sample number. Use the feature extractor to convert the candidate samples in the text dataset into vector representations. S2. Construct a sub-modulus objective function that jointly optimizes sample diversity and representativeness: （1） （2） In formula (1)-(2), For the candidate core set, For unlabeled text datasets, To indicate the budget; For the total score function; This is a normalized submodulus diversity evaluation function based on determinant point processes. For the representative evaluation function of the local density sub-modules of process awareness, An adaptive balance factor between diversity and representativeness; S3. Initialize the candidate core set to an empty set. Based on a greedy algorithm, iteratively select the candidate sample with the highest sub-modulus objective function score in each joint optimization of sample diversity and representativeness, and add it to the candidate core set. Until the scale reaches the marked budget; S4. Output the final core set after the iteration is completed and hand it over to experts for annotation.

2. The unsupervised core set selection method for context learning according to claim 1, characterized in that, The construction of the sub-model objective function for jointly optimizing sample diversity and representativeness specifically includes: S21. Normalized Submodulus Diversity Evaluation Function Based on Determinant-Based Point Processes ; （3） In formula (3), It is a matrix, whose first... i Line 1 j The column elements are the selected samples. kernel similarity, For the i-th selected sample, For the first j One selected sample; Dataset Kernel similarity matrix between candidate samples; For a custom offset, Represents the identity matrix. For the entire dataset The largest eigenvalue of the kernel matrix; the numerator of formula (3) is the added offset. The logarithmic determinant term; the denominator of formula (3) is the theoretical upper bound of DPP, used for normalization; S22, Representative Evaluation Function of Local Density Submodulus for Process Awareness ; Define a neighborhood range, i.e., the distance radius. Based on this, firstly, density scoring is defined. Measure the distance to a candidate sample The number of other samples within the sample; secondly, define the redundancy decay factor. Penalize samples that are too close to the selected sample; combine density scoring. and redundancy decay factor Build a unified : （4） Then the sample is used to its first The radius is estimated using the median of the nearest neighbor distance. ,Will With budget Perform reverse scaling, settings Candidate samples are indivual, This is a scaling factor used to determine the neighborhood size parameter. Let the value of be . For dataset The Middle i Sample Embedded vector, for No. The nearest neighbor embedding vector: （5） For median calculation, based on ,index Measure the selected sample The number of unlabeled samples in the surrounding area, while Calculated from the current core set; （6） （7） In formulas (6)-(7), express Any sample in the middle, express The sample with the highest density in the middle, These are their embedding vectors; This represents the other selected samples in the current candidate core set S. The embedding vector; This is an indicator function.

3. The unsupervised core set selection method for context learning according to claim 1, characterized in that, The adaptive estimation of the balance factor specifically includes: （8） In formula (8), the first term One term is budget-related; the expected ratio of the numerator to the denominator in the second term serves as an amplitude scaling term to balance the diversity gain. With representative gain Typical amplitude; The budget adaptive item The value depends on the current budget. With dataset size The ratio, and through a constant C Define the compensation coefficient: （9） The amplitude scaling term By performing a greedy probe to estimate the typical amplitude ratio of diversity gain to representative gain, the greedily selected probe obtains the average value, specifically: Without introducing a balance factor Under these conditions, based on the optimization objective, execute One greedy step to satisfy Record each greedy step Diversity gain and representative gain Calculate the expected ratio of the two as the amplitude ratio. ; At every step In the middle, let diversity gain be set , indicating the first Step-by-step selected sample set No. Step-by-step selected sample set diversity function difference; Similarly, representative gain Defined as .

4. The unsupervised core set selection method for context learning according to claim 1, characterized in that, S3 specifically includes: S3.1 Initialize Selected Sets It is an empty set; S3.2 proceed The greedy selection process involves executing the constructed joint optimization objective function for sample diversity and representativeness to obtain the total score for each candidate sample. The sample with the highest total score in each iteration is then added to the core candidate set. ; In the During the greedy process, From currently unselected candidate samples middle, For an unlabeled text dataset, select the sample with the largest marginal gain: （10） （11） In formula (10)-(11), For the front The selected core set after the first step For the current number The optimal sample selected in the first step. For the first The selected core set after the first step; Repeat the above process until To obtain the final core set .

5. The unsupervised core set selection method for context learning according to claim 4, characterized in that, Balance factor Setting the core set to a fixed positive value ensures the objective function... Sub-model properties.

6. The unsupervised core set selection method for context learning according to claim 3, characterized in that, The .

7. An apparatus for implementing an unsupervised core set selection method for context learning as described in any one of claims 1-6, characterized in that, The device includes: processor; A memory on which computer programs that can run on the processor are stored; The computer program, when executed by the processor, implements a step of an unsupervised core set selection method for context learning.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images