Method and device for acquiring model training sample

Abstract

The embodiment of the invention provides a method for obtaining a model training sample, and the method comprises the steps: determining a plurality of candidate representations corresponding to a first representation from a vector database based on the first representation of a target domain sample for any target domain sample in a plurality of target domain samples, the distance between the single candidate representation and the first representation is smaller than a threshold distance, the vector database comprises a plurality of second representations corresponding to the plurality of source domain samples, and the single source domain sample comprises an image, a text or a multi-modal sample. And generating a sample set for training the model based on a plurality of candidate source domain samples corresponding to the plurality of candidate representations.

Description

Technical Field

[0001] The embodiments in this specification pertain to the field of machine learning, and in particular to a method and apparatus for obtaining model training samples. Background Technology

[0002] In the field of machine learning, cross-domain data acquisition is often required. For example, for model training tasks in a target domain with sparse data, training samples need to be obtained from a data-rich source domain. However, although the source domain may be rich in data, not all source domain data is helpful for model training in the target domain; in fact, some source domain data may even negatively impact model training performance.

[0003] Therefore, there is an urgent need for a reasonable and reliable solution to obtain effective training samples from the source domain. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for obtaining model training samples, which can efficiently select the samples that are most similar to each target domain sample as a whole from multiple source domain samples.

[0005] The first aspect of this specification provides a method for obtaining model training samples, including:

[0006] For any target domain sample among multiple target domain samples, based on the first representation of the target domain sample, several candidate representations corresponding to the first representation are determined from the vector database, and the distance between a single candidate representation and the first representation is less than a threshold distance; the vector database includes multiple second representations corresponding to multiple source domain samples; a single source domain sample includes an image, text, or multimodal sample;

[0007] Based on the candidate source domain samples corresponding to the candidate representations, a sample set is generated for training the model.

[0008] A second aspect of this specification provides an apparatus for acquiring model training samples, comprising:

[0009] The determining unit is configured to, for any target domain sample among multiple target domain samples, determine several candidate representations corresponding to the first representation from the vector database based on the first representation of the target domain sample, wherein the distance between a single candidate representation and the first representation is less than a threshold distance; the vector database includes multiple second representations corresponding to multiple source domain samples; a single source domain sample includes an image, text, or multimodal sample;

[0010] The generation unit is used to generate a sample set for training the model based on the candidate source domain samples corresponding to the candidate representations.

[0011] A third aspect of this specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0012] A fourth aspect of this specification provides a computing device including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method described in the first aspect.

[0013] This specification provides a computer program product in a fifth aspect, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0014] This specification provides a method and apparatus for obtaining model training samples through one or more embodiments. By querying a vector database for the most similar source domain sample for each target domain sample, and forming a sample set based on the source domain samples retrieved for each target domain sample, it ensures that the formed sample set is most similar to all target domain samples as a whole, thus minimizing the maximum mean difference between the formed sample set and the overall target domain samples. Furthermore, since the most similar source domain sample for each target domain sample is obtained by querying a vector database, the query complexity is greatly reduced, allowing this solution to be applied to large-scale data query scenarios. In summary, this solution can efficiently select the sample most similar to all target domain samples from multiple source domain samples. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification;

[0017] Figure 2 This diagram illustrates a method for obtaining model training samples according to one embodiment of this specification.

[0018] Figure 3 This diagram illustrates a method for obtaining model training samples in one example of this specification.

[0019] Figure 4 A schematic diagram of an apparatus for obtaining model training samples according to one embodiment of this specification is shown. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0021] As mentioned earlier, to perform model training tasks, for sparse target domains, it is necessary to obtain effective training samples from data-rich source domains. However, obtaining effective training samples from data-rich source domains is very complex and often requires a lot of manpower. Therefore, we need an automated method to automatically select a large number of source domain samples from a small number of target domain samples.

[0022] Typically, to improve sample acquisition efficiency and obtain training samples that are generally similar to the samples in each target domain, those skilled in the art generally do not consider obtaining training samples by calculating the distance between each target domain sample and each source domain sample. Instead, they treat all target domain samples as a whole (called the target domain dataset or target domain sample set) and obtain training samples by calculating the maximum mean discrepancy (MMD) between the target domain dataset and each subset of source domain data divided for each source domain sample. In other words, samples are obtained by calculating the maximum mean difference between datasets.

[0023] For example, the current mainstream data acquisition approach is based on Search and Pruning (SnP). In this approach, the distance or similarity between every two source domain samples in the source domain sample set is first calculated to understand the overall distribution and density of the source domain sample set. If the number of source domain samples in the source domain sample set is m, then calculating the distance between each pair of source domain samples requires O(m... 2 The time required is O(m). Additionally, as mentioned earlier, it's necessary to calculate the distance between the target domain dataset and each source domain subset. Therefore, it's also necessary to cluster the source domain samples to obtain each source domain subset. Clustering algorithms themselves typically have high computational complexity. Finally, this scheme also requires iterative data pruning. In each iteration, it needs to select the most relevant source domain samples for the target domain dataset. This process involves a large amount of repetitive computation, ultimately leading to a computational complexity of O(m). 3 ).

[0024] In simple terms, this cubic complexity means that if the amount of data in the source domain increases tenfold, the computation time increases 1000 times. This makes the method impractical when dealing with large-scale datasets.

[0025] Therefore, this scheme considers obtaining training samples by calculating the distance between each target domain sample and each source domain sample.

[0026] The effectiveness of the training samples obtained by calculating the distance between each target domain sample and each source domain sample is first demonstrated. That is, it is proved that the sample set obtained by this method has the smallest difference between the maximum mean of the overall sample set of each target domain and the sample set of each target domain.

[0027] Suppose X is a sample set from the source domain, and Y = {y1, y2, ..., y3} n} represents the target domain sample set. For each target domain sample y i The nearest neighbor source domain sample x determined from the vector database i The resulting sample set satisfies ||x i -y i Since ||2≤δ, we need to prove that when δ→0, we have:

[0028] First of all The MMD between Y and Y is defined as follows:

[0029]

[0030] By applying the triangle inequality, we can obtain:

[0031] ||x i -x j ||≥||y i -y j ||-||x i -y i ||-||y j -x j ||≥||y i -y j ||-2δ, (Formula 2)

[0032] Typically, δ is a small value, and ||y i -y j ||>>2δ

[0033] Therefore, we can conclude that:

[0034] ||x i -x j || 2 ≥(||y i –yj ||-2δ) 2 ,

[0035] =||y i -y j || 2 -4δ||y i -y j ||+4δ 2 (Formula 3)

[0036]

[0037] Furthermore, applying the triangle inequality again, we can obtain:

[0038] ||x i -y j ||≤||y i -y j ||+||x i -y i ||

[0039] ≤||y i -y j ||+δ, (Formula 5)

[0040] Finally, the following conclusions can be drawn:

[0041]

[0042] By summing the results of all target domain samples and the nearest neighbor source domain samples in Equation 6 and Equation 4, and setting δ→0, we can obtain:

[0043]

[0044] Since the sum of the three terms in the parentheses on the right side of the inequality is 0, the entire right side is 1 / n squared multiplied by 0, which is 0. The left side of the inequality is less than the right side, obtained by adding and subtracting formulas 1, 4, and 6. Taking the limit of δ→0 gives 0, so the term containing δ in the numerator is 0, thus yielding formula 7. The left side is less than or equal to the right side, which equals 0, so the left side is less than or equal to 0. Because the distance is non-negative, therefore... It is 0.

[0045] In summary, based on the above proof, it can be concluded that if the distance between each target domain sample and the source domain sample is infinitely close to 0, then the distance between the generated sample set as a whole and the target domain sample set is close to 0, and the generated sample set and the target domain sample set are very similar.

[0046] The above proves the effectiveness of the training samples obtained by this scheme. The time complexity of this scheme will be explained below.

[0047] Specifically, to reduce time complexity, this scheme proposes storing source domain samples using a vector database. Since vector databases are typically indexed (e.g., HNSW graphs or inverted indexes like IVF), the time complexity for retrieving the nearest neighbor sample to a target domain sample from m source domain samples can be reduced to logarithmic O(logm). Furthermore, unlike traditional methods, this scheme does not require calculating pairwise distances between every two source domain samples. Instead, for each target domain sample, it retrieves the nearest source domain sample from the vector database. If the number of target domain samples is n, and assuming that k nearest source domain samples are retrieved each time, the total time complexity is n queries multiplied by the complexity of each query. Assuming k represents the number of source domain samples retrieved each time, and m represents the total number of source domain samples, then the complexity of a single query is O(logm), and the overall algorithm complexity is O(klogm).

[0048] This logarithmic complexity means that even if the number of samples in the source domain increases dramatically, the query time increases very slowly.

[0049] The above is the inventive concept provided in this specification. Based on this inventive concept, this solution can be realized. The following is a detailed description of this solution.

[0050] Figure 1 This is a schematic diagram illustrating an implementation scenario of one of the embodiments disclosed in this specification. Figure 1 In the vector database, multiple representations V1 correspond to multiple source domain samples. Here, the source domain refers to a domain with a large number of labeled samples, while the target domain refers to the actual task domain to be solved, which typically has fewer labeled samples. Source domain samples are specific data instances within the source domain, which can be images (e.g., a single image labeled as a cat), text (e.g., a text pair consisting of question and answer text, or a single text), or multimodal samples. Images can also be in video format, and multimodal samples can be a mixture of images and text (or image-text pairs). For example, an image-text pair might contain an image of an orange cat and the corresponding text "A cute orange cat is sitting on the grass."

[0051] The aforementioned representation V1 can be a vector or matrix obtained by using a feature extractor to process the sample features (or sample features and sample labels) of the corresponding source domain samples.

[0052] Specifically, taking images as the source domain samples as an example, the aforementioned feature extractor can be implemented as a Convolutional Neural Network (CNN), a ResNet series model, an EfficientNet series model, or a Vision Transformer (ViT), etc.

[0053] Taking the ViT feature extractor implementation as an example, firstly, the cat image (e.g., 224x224) can be segmented into 16x16 patches (196 in total). Then, each patch is flattened into a vector (e.g., 16x16x3 = 768 dimensions) and mapped to the embedding space through a learnable linear layer. Global features are then extracted through multiple layers of Self-Attention. Finally, the [CLS] character of ViT is output as the image representation V1. In a more specific example, this representation V1 can be a high-dimensional vector (e.g., 768 dimensions) as follows: [0.15, -0.22, 0.87, ..., 0.45].

[0054] Taking text as the source domain sample as an example, the feature extractor mentioned above can be implemented as a Bert series model or a sentence-embedding model. When the feature extractor is implemented as a Bert series model, the [CLS] character output of the model can be used as the text representation V1. In a more specific example, assuming the text is "riverbank", then its corresponding representation V1 can be, for example, a high-dimensional vector (e.g., 768 dimensions): [0.1, -0.3, ..., 0.4].

[0055] Taking multimodal samples as an example, the feature extractor mentioned above can be implemented as a Contrastive Language–Image Pre-training (CLIP), a Large-scale Image and Noisy-text Embedding (ALIGN), or the Florence visual foundation model, etc. When the feature extractor is implemented as CLIP, the representation V1 mentioned above can include two high-dimensional (e.g., 512-dimensional) vectors corresponding to the image and the text, respectively. In a more specific example, assuming the content of the image is an orange cat and the corresponding text is "a cute orange cat sitting on the grass", then the high-dimensional vector corresponding to the image can be: [0.12, -0.05, 0.88, ..., -0.23], and the high-dimensional vector corresponding to the text can be: [0.15, -0.01, 0.82, ..., -0.19].

[0056] Specifically, for any target domain sample (shown by a circle) among multiple target domain samples, based on the representation V2 of that target domain sample, several candidate representations corresponding to representation V2 are determined from the vector database, where the distance between a single candidate representation and representation V2 is less than a threshold distance. Based on the several candidate source domain samples corresponding to the several candidate representations, a sample set for training the model is generated.

[0057] Here, "model" can refer to a large language model or other machine learning models.

[0058] As mentioned earlier, this scheme generates a sample set for training the model based on a vector database. Therefore, the vector database will be explained below.

[0059] Vector databases are database systems specifically designed for the efficient storage, indexing, and querying of high-dimensional vectors. They quickly retrieve semantically similar vectors using Approximate Nearest Neighbor (ANN) search algorithms (such as HNSW graphs and IVF), making them suitable for similarity searches of unstructured data (such as images, text, audio, and video). Unlike traditional databases, they directly process feature representations (such as vectors generated by CLIP and BERT), supporting real-time retrieval of large-scale high-dimensional vectors.

[0060] In one embodiment, the above-mentioned vector database can be constructed through the following steps:

[0061] Obtain multiple source domain samples. Process the multiple source domain samples using a feature extractor to obtain multiple representations V1 corresponding to the multiple source domain samples. Add the multiple representations V1 corresponding to the multiple source domain samples to a vector database, and create a corresponding index for the vector database.

[0062] The definitions of source domain samples, feature extractor, and representation V1 can be found above and will not be repeated here.

[0063] Vector database indexes use Approximate Nearest Neighbor (ANN) algorithms (such as HNSW, IVF, or LSH) to structurally organize multiple representations V1, quickly clustering or hierarchically grouping similar representations V1. This avoids brute-force computation during retrieval, achieving millisecond-level response times. These indexes sacrifice a small amount of precision for efficiency, supporting dynamic additions and deletions and real-time queries, making them a core mechanism for balancing speed and accuracy.

[0064] In this scheme, the indexes built for the vector database may include hierarchical navigable small world (HNSW) graphs, inverted index files (IVF), or a combination of both.

[0065] Taking the HNSW graph as an example, it can include multiple layers of NSW arranged from top to bottom with the number of nodes increasing sequentially. The bottom layer of NSW contains N nodes, and each node corresponds to a representation V1.

[0066] Taking an inverted index file (IVF) as an example, it can include multiple clusters, which are obtained by clustering multiple representations V1. Each cluster corresponds to an inverted list, which records the identifier ID of each representation V1 in the corresponding cluster.

[0067] Taking the combination of the index HNSW graph and the inverted index IVF as an example, the inverted index IVF includes multiple clusters, which are obtained by clustering multiple representations V1. Each cluster corresponds to the HNSW graph, which includes multiple NSWs arranged from top to bottom with the number of nodes increasing sequentially. The bottom NSW contains N nodes, and each node corresponds to a representation V1 in that cluster.

[0068] Specifically, the inverted index (IVF) is first used for coarse-grained region localization, and the search scope is quickly narrowed down through clustering. Then, within the selected clustered regions, the HNSW graph is used for fine-grained searching, that is, the navigation characteristics of the graph structure are used to achieve local search (explained in detail later).

[0069] Figure 2 This diagram illustrates a method for obtaining model training samples according to one embodiment of this specification. This method can be executed by any device, apparatus, platform, or cluster of devices with computing and processing capabilities. Figure 2 As shown, the method may include the following steps:

[0070] Step S202: For any target domain sample among multiple target domain samples, based on the representation V2 of the target domain sample, determine several candidate representations corresponding to the representation V2 from the vector database, wherein the distance between a single candidate representation and the representation V2 is less than a threshold distance.

[0071] The distances mentioned here may include, but are not limited to, Euclidean distance, cosine distance, Manhattan distance, or Hamming distance.

[0072] Specifically, candidate representations can be determined from the vector database using its index, which will be explained in detail below.

[0073] In the case of an indexed HNSW graph, we can start from the topmost NSW of the multi-layered NSW and perform layer search on each NSW in a top-down order until we reach the bottommost NSW. Then, we search for the nearest neighbor node of representation V2 from among N nodes that is less than the threshold distance, and use it as the candidate representation corresponding to representation V2.

[0074] The above-mentioned layer search may specifically include: taking the start node of the current layer NSW as the initial current node, searching from the current node and the neighboring nodes that are connected to the current node to find the node closest to representation V2 as the updated current node, determining the current node when the search ends as the target node, and entering the next layer NSW through the target node; and the target node is used as the start node of the next layer NSW.

[0075] When the current NSW is the topmost NSW, the aforementioned starting node can be any node selected.

[0076] In addition, the above search termination conditions may include: all n-order neighbor nodes of the starting node have been searched, where n>=1; or, the distance between the current node and representation V2 is less than the distance between the current node's neighbor nodes and representation V2.

[0077] When the index is an inverted index file (IVF), target clusters can be selected from multiple clusters by calculating the similarity between representation V2 and the centroids of multiple clusters in the IVF. Based on the target identifiers in each inverted list corresponding to each target cluster, each target representation is determined, and candidate representations corresponding to representation V2 are identified from them.

[0078] Among them, each of the above target clusters can be a cluster whose corresponding similarity is greater than the similarity threshold among multiple clusters.

[0079] In addition, for each target representation, the distance between representation V2 and each target representation can be calculated, and the target representation with a distance less than the threshold distance can be selected as the candidate representation.

[0080] When the index is a combination of an HNSW graph and an inverted index (IVF), target clusters can be selected from multiple clusters by calculating the similarity between representation V2 and the centroids of multiple clusters. For any target cluster, starting from the topmost NSW of the corresponding HNSW graph, a layer search is performed on each NSW layer in a top-down order until the bottommost NSW layer, where the nearest neighbor nodes whose distance to representation V2 is less than a threshold distance are searched from N nodes. Based on the nearest neighbor nodes corresponding to each target cluster, candidate representations corresponding to representation V2 are determined.

[0081] Similarly, for each target domain sample, several candidate representations can be determined from the vector database, thus obtaining each candidate representation.

[0082] It should be noted that this solution can greatly reduce the query complexity when querying the candidate representations corresponding to each representation V2 from the vector database based on the above index, thus making this solution applicable to large-scale data query scenarios.

[0083] Step S204: Generate a sample set for training the model based on several candidate source domain samples corresponding to several candidate representations.

[0084] It should be noted that since the above candidate representations are determined from the vector database, they correspond to the source domain samples.

[0085] That is, a sample set for training the model is generated based on the candidate source domain samples corresponding to each candidate representation determined for multiple target domain samples.

[0086] In one embodiment, each candidate source domain sample can be directly used as a training sample included in the sample set.

[0087] In another embodiment, a vector database can be used to iteratively expand each candidate source domain sample. In this case, the sample set includes not only each candidate source domain sample but also the newly added samples obtained from the expansion.

[0088] This approach utilizes a vector database to iteratively expand the samples from each candidate source domain, ensuring that the approach does not repeatedly retrieve the same hot data. This helps maintain the balance of sample distribution, meaning that it can cover multiple scenarios globally.

[0089] This completes the process of obtaining training samples for large language models or machine learning models. It should be noted that the training samples obtained above (i.e., the sample set mentioned above) can be directly used for model training, fine-tuning, or evaluation in the target domain. This effectively avoids the high-cost bottleneck of training based on the full range of source domain samples, achieving maximum cost-effectiveness in data usage.

[0090] In summary, this scheme queries the vector database for the most similar source domain sample for each target domain sample, and forms a sample set based on the source domain samples retrieved for each target domain sample. This ensures that the formed sample set is most similar to all target domain samples as a whole, thus minimizing the maximum mean difference between the formed sample set and the overall target domain samples. Furthermore, since the most similar source domain sample for each target domain sample is obtained by querying the vector database, the query complexity is significantly reduced, making this scheme applicable to large-scale data query scenarios. In conclusion, this scheme can efficiently select the sample most similar to all target domain samples from multiple source domain samples. In other words, this scheme can significantly reduce complexity while ensuring the effectiveness of the acquired training samples.

[0091] Figure 3 This document illustrates a method for obtaining model training samples in one example. Figure 3In this process, the distance between each target domain sample's representation V2 (shown as a circle filled with vertical lines) and each source domain sample's representation V1 (shown as a circle filled with horizontal lines) in the vector database is calculated to determine the candidate representations corresponding to each target domain sample's representation V2, thus forming a sample set. In other words, the source domain sample most similar to each target domain sample is obtained to form the sample set.

[0092] It should also be noted that, experimental verification shows that this solution can achieve model training results close to or exceeding those based on all source domain samples by selecting only 10%-30% of the source domain samples. Furthermore, this solution can significantly save computing power and time costs, and provides more accurate adaptation to the target domain. Finally, this solution is also easy to integrate with enterprise-level data governance platforms or middleware, meaning it has good adaptability.

[0093] Corresponding to the method for obtaining model training samples described above, one embodiment of this specification also provides an apparatus for obtaining model training samples, such as... Figure 4 As shown, the device may include:

[0094] The determining unit 402 is configured to, for any target domain sample among multiple target domain samples, determine several candidate representations corresponding to the first representation from a vector database based on the first representation of the target domain sample, wherein the distance between a single candidate representation and the first representation is less than a threshold distance. The vector database includes multiple second representations corresponding to multiple source domain samples, and a single source domain sample includes an image, text, or a multimodal sample;

[0095] The generation unit 404 is used to generate a sample set for training the model based on several candidate source domain samples corresponding to several candidate representations.

[0096] In one embodiment, the device further includes:

[0097] Acquisition unit 406 is used to acquire multiple source domain samples;

[0098] The processing unit 408 is used to process multiple source domain samples using a feature extractor to obtain multiple second representations corresponding to the multiple source domain samples.

[0099] Add unit 410 to add multiple second representations corresponding to multiple source domain samples to the vector database and establish corresponding indexes for the vector database.

[0100] In one embodiment, the determining unit 402 is specifically used for:

[0101] Based on the first representation of the target domain sample, several candidate representations corresponding to the first representation are determined from the vector database using an index.

[0102] In one embodiment, the index is a hierarchical navigable small-world HNSW graph, which includes multiple layers of NSW arranged from top to bottom with an increasing number of nodes, wherein the bottom layer NSW contains N nodes, each of which corresponds to a second representation.

[0103] The determining unit 402 is specifically used for:

[0104] Starting from the topmost NSW of the multi-layered NSW, perform layer search on each NSW in a top-down order until the bottommost NSW. Then, search for the nearest neighbor node that is less than a threshold distance from the first representation among N nodes, and use it as a candidate representation corresponding to the first representation.

[0105] In another embodiment, the index is an inverted index file (IVF) that includes multiple clusters, with each first cluster corresponding to a first inverted list, which records the identifiers of each second representation in the first cluster.

[0106] The determining unit 402 is also specifically used for:

[0107] By calculating the similarity between the first representation and the center points of multiple clusters, each target cluster is selected from multiple clusters;

[0108] Based on the target identifiers in each inverted list corresponding to each target cluster, the target representation is determined, and the candidate representation corresponding to the first representation is determined from them.

[0109] In another embodiment, the index is a combination of a hierarchical navigable small-world (HNSW) graph and an inverted index (IVF) file, which includes multiple clusters obtained by clustering multiple second representations. Any first cluster corresponds to an HNSW graph, including multiple layers of NSW arranged from top to bottom with increasing number of nodes, wherein the bottommost NSW contains N nodes, and each node corresponds to a second representation in a first cluster.

[0110] The determining unit 402 is also specifically used for:

[0111] By calculating the similarity between the first representation and the center points of multiple clusters, each target cluster is selected from multiple clusters;

[0112] For any first target cluster, starting from the topmost NSW of the multi-layered NSWs included in the corresponding first HNSW graph, perform a layer search on each NSW in order from top to bottom until in the bottommost NSW, search for the nearest neighbor node whose distance from the first representation is less than the threshold distance from N nodes.

[0113] Based on the nearest neighbor nodes corresponding to each target cluster, determine the candidate representation corresponding to the first representation.

[0114] In one embodiment, the distance described above includes any of the following:

[0115] Euclidean distance, cosine distance, Manhattan distance, and Hamming distance.

[0116] In one embodiment, the above model includes a large language model or a machine learning model.

[0117] The functions of each functional unit of the apparatus in the above embodiments of this specification can be implemented through the steps of the above method embodiments. Therefore, the specific working process of the apparatus provided in one embodiment of this specification will not be repeated here.

[0118] The apparatus for obtaining model training samples provided in one embodiment of this specification can efficiently select samples that are most similar to each target domain sample as a whole from multiple source domain samples.

[0119] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a combination Figure 2 The method described.

[0120] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements a combination... Figure 2 The method described.

[0121] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the medium or device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0122] The steps of the methods or algorithms described in conjunction with the disclosure in this specification can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, external hard disk, CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a server. Of course, the processor and storage medium can also exist as discrete components in the server.

[0123] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0124] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26k20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0125] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0126] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes the elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0127] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0128] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0131] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0132] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0133] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0134] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0136] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0137] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A method for obtaining model training samples, comprising: For any target domain sample among multiple target domain samples, based on the first representation of the target domain sample, several candidate representations corresponding to the first representation are determined from the vector database, and the distance between a single candidate representation and the first representation is less than a threshold distance. The vector database includes multiple second representations corresponding to multiple source domain samples; a single source domain sample includes an image, text, or a multimodal sample; Based on the candidate source domain samples corresponding to the candidate representations, a sample set is generated for training the model.

2. The method according to claim 1, wherein, The vector database is obtained through the following steps: Obtain the multiple source domain samples; The multiple source domain samples are processed by a feature extractor to obtain multiple second representations corresponding to the multiple source domain samples; Multiple second representations corresponding to the multiple source domain samples are added to the vector database, and a corresponding index is created for the vector database.

3. The method according to claim 1, wherein, The first representation based on the target domain sample determines several candidate representations corresponding to the first representation from the vector database, including: Based on the first representation of the target domain sample, several candidate representations corresponding to the first representation are determined from the vector database using an index.

4. The method according to claim 3, wherein, The index is a hierarchical navigable small-world (HNSW) graph; the HNSW graph includes multiple layers of NSW arranged from top to bottom with the number of nodes increasing sequentially; the bottom layer of NSW contains N nodes, each of which corresponds to a second representation; The method of using an index to determine several candidate representations corresponding to the first representation from the vector database includes: Starting from the topmost NSW of the multi-layered NSW, layer search is performed on each NSW in a top-to-bottom order until the bottommost NSW. Then, from the N nodes, the nearest neighbor nodes whose distance to the first representation is less than a threshold distance are searched as several candidate representations corresponding to the first representation.

5. The method according to claim 3, wherein, The index is an inverted index file (IVF); the IVF includes multiple clusters, which are obtained by clustering the multiple second representations; any first cluster corresponds to a first inverted list, which records the identifiers of each second representation in the first cluster; The method of using an index to determine several candidate representations corresponding to the first representation from the vector database includes: By calculating the similarity between the first representation and the center point of the plurality of clusters, each target cluster is selected from the plurality of clusters; Based on each target identifier in each inverted list corresponding to each target cluster, each target representation is determined, and several candidate representations corresponding to the first representation are determined from them.

6. The method according to claim 3, wherein, The index is a combination of a hierarchical navigable small-world (HNSW) graph and an inverted index (IVF); the IVF includes multiple clusters, which are obtained by clustering the multiple second representations; any first cluster corresponds to an HNSW graph, which includes a multi-layered NSW arranged from top to bottom with an increasing number of nodes; wherein the bottommost NSW contains N nodes, each of which corresponds to a second representation in a first cluster. The method of using an index to determine several candidate representations corresponding to the first representation from the vector database includes: By calculating the similarity between the first representation and the center point of the plurality of clusters, each target cluster is selected from the plurality of clusters; For any first target cluster, starting from the topmost NSW of the multi-layered NSWs included in the corresponding first HNSW graph, perform a layer search on each NSW in order from top to bottom until in the bottommost NSW, search for the nearest neighbor node whose distance from the first representation is less than the threshold distance from the N nodes. Based on the nearest neighbor nodes corresponding to each target cluster, several candidate representations corresponding to the first representation are determined.

7. The method according to claim 1, wherein, The distance includes any of the following: Euclidean distance, cosine distance, Manhattan distance, and Hamming distance.

8. The method according to claim 1, wherein, The model includes large language models or machine learning models.

9. An apparatus for acquiring model training samples, comprising: The determining unit is used to determine, for any target domain sample among multiple target domain samples, several candidate representations corresponding to the first representation from the vector database based on the first representation of the target domain sample, wherein the distance between a single candidate representation and the first representation is less than a threshold distance. The vector database includes multiple second representations corresponding to multiple source domain samples; a single source domain sample includes an image, text, or a multimodal sample; The generation unit is used to generate a sample set for training the model based on the candidate source domain samples corresponding to the candidate representations.

10. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-8.

Images