Sample determination method and device, electronic equipment and storage medium
By determining the target sampling points in high-dimensional vector space and sampling, the problem of insufficient sample generation richness in natural language processing is solved. The generated samples are diversified and the model performance is improved, and no dependence on domain knowledge is achieved.
Patent Information
- Application Number
- CN202411188180.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-12-31
AI Technical Summary
The existing technology lacks the richness of samples generated in natural language processing, which is difficult to meet the needs of diversified samples, and has a high dependence on domain knowledge, resulting in limited improvement in model performance.
By converting the to-processed data into high-dimensional vectors, the target sampling points are determined in the high-dimensional vector space and sampled, the sample data is obtained, and a diverse sample is generated. This method requires no domain knowledge, and the target sampling point position is controllable, so it can be customized according to needs.
The generated samples are more diverse, and the target sampling points are controllable during the mining process, which improves model performance and allows rich samples to be obtained without domain knowledge, which improves the interpretability and versatility of the method.
Smart Images
Figure CN119227668A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data mining technology, and in particular to a sample determination method, device, electronic device, and storage medium. Background Art
[0002] With the continuous development of natural language processing technology, improving model performance has become a key issue. As a powerful and effective solution to improve model performance, data augmentation plays an important role in natural language processing tasks.
[0003] However, if the generated samples are not rich enough, it will be difficult to meet the demand for diverse samples in natural language processing tasks. Moreover, they rely on domain knowledge and require technicians to be very familiar with domain knowledge. Summary of the Invention
[0004] The present disclosure provides a sample determination method, device, electronic device, and storage medium to at least solve the above technical problems existing in the prior art.
[0005] According to a first aspect of the present disclosure, a sample determination method is provided, comprising:
[0006] Determine the high-dimensional vector corresponding to the data to be processed;
[0007] Determining a target sampling point based on the high-dimensional vector and the number of categories of the high-dimensional vector;
[0008] Based on the target sampling point, determining at least one sampling vector in a vector space;
[0009] Confirm that the sampling data corresponding to the at least one sampling vector is a sample of the data to be processed.
[0010] In the above solution, the determination of the vector corresponding to the data to be processed is implemented by a vector encoder, and the method further includes:
[0011] Training the vector encoder based on the data training set so that the distance between high-dimensional vectors corresponding to training data of the same category in the vector space is less than a first threshold, and the distance between high-dimensional vectors corresponding to training data of different categories in the vector space is greater than a second threshold;
[0012] Determining the high-dimensional vector corresponding to the data to be processed includes:
[0013] The data to be processed is input into a vector encoder, and it is confirmed that the output of the vector encoder is a high-dimensional vector corresponding to the data to be processed.
[0014] In the above solution, determining the target sampling point based on the high-dimensional vector and the number of categories of the high-dimensional vector includes one of the following:
[0015] In response to the number of categories of the data to be processed being 1, the target sampling point is the location of the high-dimensional vector;
[0016] In response to the number of categories of the data to be processed being 2, the target sampling point is on a line corresponding to two high-dimensional vectors;
[0017] Among them, if the target sampling point is on the connecting line corresponding to the two high-dimensional vectors, and the category corresponding to the sampling vector is close to the category of the first high-dimensional vector in the two high-dimensional vectors, then the distance between the target sampling point and the first high-dimensional vector is the shortest.
[0018] In the above solution, determining the target sampling point based on the high-dimensional vector corresponding to the data to be processed includes:
[0019] In response to the number of categories of the data to be processed being greater than or equal to 3, the target sampling point is within a region enclosed by the high-dimensional vector corresponding to the data to be processed;
[0020] If the category corresponding to the sampling vector is close to the category of the second high-dimensional vector in the high-dimensional vector corresponding to the data to be processed, the distance between the target sampling point and the second high-dimensional vector is the shortest.
[0021] In the above solution, in response to the number of categories of the data to be processed being 1, in the vector space, the at least one sampling vector is located within a sphere with the high-dimensional vector as the center and the preset distance as the radius;
[0022] In response to the number of categories of the data to be processed being 2, in the vector space, the at least one sampling vector is located in the space between the two high-dimensional vectors;
[0023] In response to the number of categories of the data to be processed being greater than or equal to 3, in the vector space, the at least one sampling vector is located in the space between the high-dimensional vectors corresponding to the data to be processed.
[0024] In the above solution, confirming that the sampled data corresponding to the at least one sampling vector is a sample of the data to be processed includes:
[0025] The at least one sampling vector is input to a vector decoder, and an output of the vector decoder is confirmed to be a sample of the data to be processed.
[0026] In the above solution, after confirming that the sampled data corresponding to the at least one sampling vector is a sample of the data to be processed, the method further includes:
[0027] Filtering samples of the data to be processed based on the first condition, and confirming that the filtered samples are target samples of the data to be processed;
[0028] The first condition includes at least one of the same category, the same expression and semantic coherence.
[0029] According to a second aspect of the present disclosure, a sample determination device is provided, the device comprising:
[0030] A vector encoding unit, used to determine the high-dimensional vector corresponding to the data to be processed;
[0031] a target sampling point confirmation unit, configured to determine a target sampling point based on the high-dimensional vector and the number of categories of the high-dimensional vector;
[0032] a sampling unit, configured to determine at least one sampling vector in a vector space based on the target sampling point;
[0033] The vector decoding unit is used to confirm that the sampling data corresponding to the at least one sampling vector is a sample of the data to be processed.
[0034] According to a third aspect of the present disclosure, there is provided an electronic device, including:
[0035] at least one processor; and
[0036] a memory communicatively connected to the at least one processor; wherein,
[0037] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.
[0038] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.
[0039] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings, in which several embodiments of the present disclosure are shown by way of example and not limitation, wherein:
[0041] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.
[0042] Figure 1 A first optional flow chart of the sample determination method provided by an embodiment of the present disclosure is shown;
[0043] Figure 2 A second optional flow chart of the sample determination method provided by an embodiment of the present disclosure is shown;
[0044] Figure 3 A third optional flow chart of the sample determination method provided in an embodiment of the present disclosure is shown;
[0045] FIG4( a ) shows a first schematic diagram of determining a target sampling point provided by an embodiment of the present disclosure;
[0046] FIG4( b ) shows a second schematic diagram of determining a target sampling point provided by an embodiment of the present disclosure;
[0047] FIG4( c ) shows a third schematic diagram of determining a target sampling point provided by an embodiment of the present disclosure;
[0048] Figure 5 A schematic diagram of the sampling effect provided by an embodiment of the present disclosure is shown;
[0049] Figure 6 A fourth optional flow chart of the sample determination method provided in an embodiment of the present disclosure is shown;
[0050] Figure 7 A schematic diagram of an optional process for obtaining samples corresponding to data to be processed provided by an embodiment of the present disclosure is shown;
[0051] Figure 8 An optional structural diagram of a sample determination device provided by an embodiment of the present disclosure is shown;
[0052] Figure 9 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0053] To make the purposes, features, and advantages of the present disclosure more apparent and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative work shall fall within the scope of protection of the present disclosure.
[0054] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0055] In the following description, the terms "first\second" are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.
[0056] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by those skilled in the art in the art of this disclosure. The terms used in this disclosure are only for the purpose of describing the embodiments of this disclosure and are not intended to limit this disclosure.
[0057] It should be understood that in the various embodiments of the present disclosure, the size of the serial number of each implementation process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.
[0058] From the perspective of model performance, some samples, such as difficult samples, are those whose categories the model still finds difficult to correctly predict even after multiple training iterations. From the perspective of feature space, the features corresponding to the data to be processed are in the fuzzy area between the boundaries of different categories. Their features have certain similarities with multiple categories, making it difficult for the classification model to accurately determine the category to which they belong. In order to improve the performance of the classification model for difficult samples, data enhancement is required for difficult samples. There are three main methods for mining difficult samples in related technologies:
[0059] (1) Using manual rule synthesis methods. However, this method requires high human resources and certain domain knowledge.
[0060] (2) Using language model generation. However, the generated samples are not rich enough to improve the model performance.
[0061] (3) Combined with a large language model, it can be guided to generate difficult samples through instructions. However, the generated samples are also light and lack richness, and the instructions require manual intervention and are dependent on domain knowledge.
[0062] In response to the defects existing in the related art, the embodiments of the present disclosure provide a sample determination method, which converts data into high-dimensional vectors, performs vector sampling in the high-dimensional vector space, and finally converts the sampled sampling vectors into data to obtain sample data.
[0063] Figure 1 A first optional flow chart of the sample determination method provided in an embodiment of the present disclosure is shown, and will be explained according to each step.
[0064] Step S101: Determine the high-dimensional vector corresponding to the data to be processed.
[0065] In some embodiments, the data to be processed may be a difficult sample or a non-difficult sample.
[0066] In some embodiments, a carrier implementing the sample determination method (hereinafter referred to as carrier) determines a high-dimensional vector corresponding to the data to be processed in a vector space. The vector space is a high-dimensional vector space.
[0067] Among them, the carrier can be a computer program, electronic circuit, database, mobile application, electronic device, cloud computing platform, distributed system, artificial intelligence framework, mathematical model, automation tool and microcontroller, etc., which can realize software or hardware of algorithm and method process.
[0068] Step S102 : determining target sampling points based on the high-dimensional vector and the number of categories of the high-dimensional vector.
[0069] In some embodiments, the data to be processed may include multiple categories or one category; the carrier determines the position of the target sampling point in the vector space based on the position of the high-dimensional vector in the vector space and the number of categories corresponding to the data to be processed.
[0070] In some embodiments, the number of categories of the high-dimensional vector is different, the target sampling point is confirmed in different ways, and the position of the target sampling point in the vector space is determined based on the position of the high-dimensional vector in the vector space.
[0071] Step S103: determining at least one sampling vector in a vector space based on the target sampling point.
[0072] In some embodiments, the carrier samples in a vector space based on the target sampling point to determine at least one sampling vector. Optionally, in the vector space, the distance between any one of the at least one sampling vectors and the target sampling point is less than a third threshold. The third threshold can be set based on actual needs.
[0073] Step S104 : confirming that the sampling data corresponding to the at least one sampling vector is a sample of the data to be processed.
[0074] In some embodiments, the carrier determines the sampling data corresponding to each sampling vector, and confirms that the sampling data is a sample of the data to be processed.
[0075] In this way, through the sample determination method provided by the embodiment of the present disclosure, sampling is performed in a high-dimensional vector space, and then the sampling vectors obtained by sampling are converted into samples, so that the mined samples can be more diversified; and in the mining process, the position of the target sampling point is controllable, and can be customized according to needs. Taking the intent recognition task as an example, the distance ratio of the sampling vector to different categories can be quantitatively controlled; in addition, the method provided by the embodiment of the present disclosure is more interpretable and versatile, and rich samples can be obtained without domain knowledge.
[0076] Figure 2 A second optional flow chart of the sample determination method provided in an embodiment of the present disclosure is shown, and will be explained according to each step.
[0077] Step S201: training a vector encoder based on a data training set.
[0078] In some embodiments, the carrier labels the training data in the data training set and determines the type of each training data; taking the training data as text as an example, the type can include politics, economy, sports and entertainment, etc.; it can also include information retrieval, problem solving, creative inspiration, analysis and evaluation, language relevance and intention recognition, etc.; it can also include knowledge expansion, decision-making assistance, historical and cultural exploration, scientific and technological frontier tracking, emotional communication and entertainment and leisure, etc.
[0079] In some optional embodiments, the carrier may also pre-process the training data in the data training set, including operations such as word segmentation, stop word removal, and secondary stem extraction.
[0080] In some embodiments, the carrier inputs training data from a data training set into a vector encoder and determines the position of a high-dimensional sample vector of each training data in a vector space. The purpose of the training is to make the distance between high-dimensional vectors corresponding to training data of the same category in the vector space less than a first threshold, and to make the distance between high-dimensional vectors corresponding to training data of different categories in the vector space greater than a second threshold. That is, the high-dimensional sample vectors corresponding to training data of the same type are closer in the vector space, and the high-dimensional sample vectors corresponding to training data of different types are farther apart in the vector space. The loss function is determined based on the principle that the high-dimensional sample vectors corresponding to training data of the same type are closer in the vector space, and the high-dimensional sample vectors corresponding to training data of different types are farther apart in the vector space. The loss function can be a contrast loss function or a triplet loss function.
[0081] If the loss function is a triplet loss function, the vector encoder is optimized by constructing a triplet (anchor sample, positive sample, negative sample) so that the distance between the anchor sample and the positive sample is smaller than the distance between the anchor sample and the negative sample.
[0082] Step S202: input the data to be processed into a vector encoder to obtain a high-dimensional vector.
[0083] In some embodiments, the carrier inputs the data to be processed into a vector encoder to obtain a high-dimensional vector of the data to be processed in a vector space.
[0084] Step S203 : determining target sampling points based on the high-dimensional vector and the number of categories of the high-dimensional vector.
[0085] The specific steps of step S203 are the same as those of step S102 and will not be repeated here.
[0086] Step S204: determining at least one sampling vector in a vector space based on the target sampling point.
[0087] The specific steps of step S204 are the same as those of step S103 and will not be repeated here.
[0088] Step S205 : confirming that the sampling data corresponding to the at least one sampling vector is a sample of the data to be processed.
[0089] In some embodiments, the carrier inputs the at least one sampling vector into a vector decoder to obtain sampled data, where each sampling vector corresponds to one sampled data. The vector decoder is configured to convert vectors in a vector space into text data. The sampled data is embodied in text form.
[0090] In some embodiments, the carrier determines at least one sampled data as a sample of the data to be processed.
[0091] In some optional embodiments, the carrier may also filter samples of the data to be processed based on a first condition to confirm that the filtered samples are target samples of the data to be processed; the first condition includes at least one of the same category, the same expression, and semantic coherence.
[0092] In this way, through the sample determination method provided by the embodiment of the present disclosure, the data to be processed (i.e., the data to be mined) is converted into a high-dimensional space, sampled in the high-dimensional vector space, and then the sampling vectors obtained by sampling are converted into samples, so that the mined samples can be more diversified; and during the mining process, the position of the target sampling point is controllable and can be customized according to needs; in addition, the method provided by the embodiment of the present disclosure is more interpretable and versatile, and rich samples can be obtained without domain knowledge.
[0093] Figure 3 A third optional flow chart of the sample determination method provided in an embodiment of the present disclosure is shown, and will be explained according to each step.
[0094] Step S301: Determine the high-dimensional vector corresponding to the data to be processed.
[0095] In some embodiments, the carrier may input the data to be processed into the vector encoder trained in step S201 to obtain a corresponding high-dimensional vector.
[0096] In some embodiments, the data to be processed may be at least one data of the same category, i.e., at least one data of the same category, or at least two data of different categories, i.e., at least two of the at least two data have different categories.
[0097] Step S302 : determining target sampling points based on the position of the high-dimensional vector in the vector space and the number of categories of the high-dimensional vector.
[0098] In some embodiments, the number of categories of the data to be processed is different, and the method of determining the target sampling points is different.
[0099] FIG4( a ) shows a first schematic diagram of determining a target sampling point provided by an embodiment of the present disclosure.
[0100] In some optional embodiments, as shown in FIG4(a), if the number of categories of the data to be processed is 1, that is, the data to be processed is at least one data of the same category, then the target sampling point is the location of the high-dimensional vector (the circle surrounded by the dotted line in FIG4(a)). Specifically, if the number of data to be processed is 1, then the target sampling point is the location of the high-dimensional vector; if the number of data to be processed is at least two, then the target sampling point can be the location of any high-dimensional vector, or can be the center of a graph surrounded by at least two high-dimensional vectors, that is, the point in the vector space with the shortest sum of distances to each high-dimensional vector is the target sampling point.
[0101] FIG4( b ) shows a second schematic diagram of determining target sampling points provided by an embodiment of the present disclosure.
[0102] In some other optional embodiments, as shown in FIG4(b), if the number of categories of the data to be processed is 2, that is, the data to be processed includes at least two data of two types, then the target sampling point is on the connecting line corresponding to the two categories. The connecting line corresponding to the two categories, that is, each solid circle in FIG4(b) is the center point of a category, the dotted line between the two solid circles is the connecting line corresponding to the two categories, and the circle surrounded by the dotted line on the connecting line is the target sampling point.
[0103] Specifically, if the high-dimensional vector of each category is 1, that is, the data to be processed includes two data, and the two data types are different, then the target sampling point is on the line corresponding to the two high-dimensional vectors; alternatively, if the number of high-dimensional vectors in each category is two or more, then a first center point of the first category and a second center point of the second category are determined; the first center point can be the position corresponding to any high-dimensional vector in the category, or can be the center of a graph surrounded by two or more high-dimensional vectors of the first category, that is, the point in the vector space with the shortest sum of distances to each high-dimensional vector. The second center point is determined in the same manner as the first center point, and will not be repeated here. The target sampling point is determined to be on the line between the first center point and the second center point.
[0104] Optionally, if the first category includes one high-dimensional vector and the second category includes at least two high-dimensional vectors, a second center point of the second category is determined, and the target sampling point is determined to be on the line between the high-dimensional vector corresponding to the first category and the second center point.
[0105] In some embodiments, if the target sampling point is on the connecting line and the category corresponding to the sampling vector is close to the first category, then the distance between the target sampling point and the high-dimensional vector or first center point corresponding to the first category is the shortest, i.e., the target sampling point is closer to the high-dimensional vector or first center point corresponding to the first category. As shown in Figure 4(b), the distance between the circle surrounded by the dotted line and the solid circle near the far point is shorter, i.e., the category of the target sampling point is similar or the same as the category corresponding to the solid circle near the far point.
[0106] In some optional embodiments, if the mined samples are applied to an intent recognition task, the distance ratios between the target sampling points and different categories can be controlled.
[0107] FIG4( c ) shows a third schematic diagram of determining target sampling points provided by an embodiment of the present disclosure.
[0108] In some further optional embodiments, in response to the number of categories of the data to be processed being greater than or equal to 3, the target sampling point is within a region enclosed by high-dimensional vectors corresponding to the data to be processed.
[0109] Specifically, as shown in FIG4 (c), for a category including at least two high-dimensional vectors, when determining the center point, the location of any high-dimensional vector in the category can be used as the location of the center point. Alternatively, the center point can be determined based on the center of the graph surrounded by two or more high-dimensional vectors in the category (the circle surrounded by the dotted line in FIG4 (c)), that is, the point in the vector space with the shortest sum of distances to each high-dimensional vector. For a category with only one high-dimensional vector, its center point is the location of the high-dimensional vector. Furthermore, the carrier determines the target sampling point based on the area surrounded by the center point of each type. If the category corresponding to the sampling vector is the same as or similar to the category of the second high-dimensional vector in the high-dimensional vector corresponding to the data to be processed, then the distance between the target sampling point and the second high-dimensional vector is the shortest, that is, the target sampling point is closer to the second high-dimensional vector.
[0110] Alternatively, the carrier may also determine at least two target sampling points based on the center points of any two categories, and the category of each target sampling point is different from that of the other target sampling points.
[0111] Step S303: Determine at least one sampling vector in a vector space based on the target sampling point.
[0112] In some embodiments, in response to the number of categories of the data to be processed being 1, in the vector space, the at least one sampling vector is located within a sphere with the high-dimensional vector as the center and a preset distance as the radius; that is, the sampling is performed within a super-hollow sphere with the high-dimensional vector as the center and a preset radius.
[0113] In other embodiments, in response to the number of categories of the data to be processed being 2, in the vector space, the at least one sampling vector is located in the space between two high-dimensional vectors, that is, sampling is performed in a super hollow sphere with the target sampling point as the center of the sphere and the preset distance as the radius to obtain at least one sampling vector.
[0114] In some embodiments, in response to the number of categories of the data to be processed being greater than or equal to 3, in the vector space, the at least one sampling vector is located in the space between the high-dimensional vectors corresponding to the data to be processed; that is, sampling is performed within a super-hollow sphere with the target sampling point as the center of the sphere and the preset distance as the radius to obtain at least one sampling vector.
[0115] Figure 5 A schematic diagram of the sampling effect provided by an embodiment of the present disclosure is shown.
[0116] For visualization, the vector is subjected to dimensionality reduction processing. Assume that the data to be processed includes data of four categories. The high-dimensional vectors corresponding to each category are clustered together in the vector space. During sampling, a target sampling point can be determined based on the high-dimensional vectors of the four categories, with the line connecting every two categories. Five target sampling points are determined, and sampling is performed based on the five target sampling points to obtain high-dimensional vectors corresponding to the five categories.
[0117] Step S304: confirm that the sampling data corresponding to the at least one sampling vector is a sample of the data to be processed.
[0118] In some embodiments, the carrier inputs the at least one sampling vector into a vector decoder to obtain sampled data, where each sampling vector corresponds to one sampled data. The vector decoder is configured to convert vectors in a vector space into text data. The sampled data is embodied in text form.
[0119] In some embodiments, the carrier determines at least one sampled data as a sample of the data to be processed.
[0120] In some optional embodiments, the carrier may also filter samples of the data to be processed based on a first condition to confirm that the filtered samples are target samples of the data to be processed; the first condition includes at least one of the same category, the same expression, and semantic coherence.
[0121] In this way, through the sample determination method provided by the embodiment of the present disclosure, the data to be processed (i.e., the data to be mined) is converted into a high-dimensional space, sampled in the high-dimensional vector space, and then the sampling vectors obtained by sampling are converted into samples, so that the mined samples can be more diversified; and during the mining process, the position of the target sampling point is controllable and can be customized according to needs; in addition, the method provided by the embodiment of the present disclosure is more interpretable and versatile, and rich samples can be obtained without domain knowledge.
[0122] Figure 6 A fourth optional flow chart of the sample determination method provided in an embodiment of the present disclosure is shown, and will be explained according to each step.
[0123] Step S601: training a vector encoder and a vector decoder.
[0124] In some embodiments, text sequences (or text data) from different fields (or categories) are collected to form a data training set, and the vector encoder and vector decoder are trained based on the data training set.
[0125] Among them, the vector encoder is used to convert the text sequence into a high-dimensional vector, and the high-dimensional vectors corresponding to related or similar text sequences are close in the vector space, and the high-dimensional vectors corresponding to unrelated text sequences are far apart in the vector space. The specific training method is the same as step S201 and will not be repeated here.
[0126] In some embodiments, the vector decoder is used to convert a high-dimensional vector into a text sequence. After training the vector encoder, the carrier inputs the data in the data training set into the vector encoder to obtain a high-dimensional vector, and inputs the high-dimensional vector into the vector decoder to obtain a predicted text sequence; a cross-entropy loss function is determined based on the text sequence and the predicted text sequence in the data training set, and the parameters of the vector decoder are adjusted based on the cross-entropy loss function.
[0127] Step S602: Sampling the high-dimensional vector of the data to be processed to obtain a sampling vector, and obtaining a text sequence based on the sampling vector.
[0128] In some embodiments, a high-dimensional vector corresponding to the data to be processed is obtained based on the vector encoder, and sampling is performed in the vector space to obtain a sampling vector; the sampling vector is input into a vector decoder to obtain sampling data corresponding to the sampling vector, and the sampling data is a text sequence.
[0129] In some optional embodiments, the carrier may select sample points from the data to be processed. Specifically, the carrier may select sample points based on the classification results of the text sequences of each category in the classification task, and the carrier may select data whose classification results differ from the annotations of the text sequences as sample points. For each category, the carrier may select one data point as a sample point, or may select multiple data points as sample points.
[0130] In some optional embodiments, the carrier determines a target sampling point using a high-dimensional vector corresponding to a sample point, performs sampling around the target sampling point, and obtains at least one sampling vector.
[0131] Specifically, if the carrier selects one sample point, the high-dimensional vector corresponding to the sampling point is used as the target sampling point; if the carrier selects two sample points, any point on the line connecting the high-dimensional vectors corresponding to the two sample points is used as the target sampling point. Optionally, the target sampling point is closer to the high-dimensional vector corresponding to the mined difficult sample; if the carrier selects three or more sample points, any point inside the geometric figure surrounded by the three or more sample points is used as the target sampling point.
[0132] Step S603: Filter the text sequence to obtain samples corresponding to the data to be processed.
[0133] Figure 7A schematic diagram of an optional process for obtaining samples corresponding to data to be processed provided by an embodiment of the present disclosure is shown.
[0134] In some optional embodiments, such as Figure 7 As shown, the carrier obtains the sampling data corresponding to the sampling vector based on the vector decoder, that is, vector to text (Vector2Text), and then inputs the sampling data into the large language model for filtering (LLMFilter), deletes the sample data with incoherent sentences or semantics that are not close to the data to be processed, and finally obtains the sample corresponding to the data to be processed, that is, the sample labeled the same as the data to be processed (LLM Tagging).
[0135] In this way, through the sample determination method provided by the embodiment of the present disclosure, the data to be processed (i.e., the data to be mined) is converted into a high-dimensional space, sampled in the high-dimensional vector space, and then the sampled vectors are converted into sample data, and then filtered through a large language model to obtain target samples, which can make the mined samples more diversified; and during the mining process, the position of the target sampling point is controllable and can be customized according to needs; in addition, the method provided by the embodiment of the present disclosure is more interpretable and versatile, and rich samples can be obtained without domain knowledge.
[0136] Figure 8 An optional structural diagram of a sample determination device provided in an embodiment of the present disclosure is shown, and will be described according to each part.
[0137] In some embodiments, the sample determination device 700 includes a vector encoding unit 701 , a target sampling point confirmation unit 702 , a sampling unit 703 and a vector decoding unit 704 .
[0138] The vector encoding unit 701 is used to determine the high-dimensional vector corresponding to the data to be processed;
[0139] The target sampling point confirmation unit 702 is configured to determine the target sampling point based on the high-dimensional vector and the number of categories of the high-dimensional vector;
[0140] The sampling unit 703 is configured to determine at least one sampling vector in a vector space based on the target sampling point;
[0141] The vector decoding unit 704 is configured to confirm that the sampling data corresponding to the at least one sampling vector is a sample of the data to be processed.
[0142] The vector encoding unit 701 is implemented based on a vector encoder, and is further configured to: train the vector encoder based on a data training set so that the distance between high-dimensional vectors corresponding to training data of the same category in the vector space is less than a first threshold, and the distance between high-dimensional vectors corresponding to training data of different categories in the vector space is greater than a second threshold;
[0143] The vector encoding unit 701 is specifically configured to input the data to be processed into a vector encoder, and confirm that the output of the vector encoder is a high-dimensional vector corresponding to the data to be processed.
[0144] The target sampling point confirmation unit 702 is specifically configured to:
[0145] In response to the number of categories of the data to be processed being 1, the target sampling point is the location of the high-dimensional vector;
[0146] In response to the number of categories of the data to be processed being 2, the target sampling point is on a line corresponding to two high-dimensional vectors;
[0147] Among them, if the target sampling point is on the connecting line corresponding to the two high-dimensional vectors, and the category corresponding to the sampling vector is close to the category of the first high-dimensional vector in the two high-dimensional vectors, then the distance between the target sampling point and the first high-dimensional vector is the shortest.
[0148] The target sampling point confirmation unit 702 is specifically configured to, in response to the number of categories of the data to be processed being greater than or equal to 3, determine that the target sampling point is within a region enclosed by the high-dimensional vector corresponding to the data to be processed;
[0149] If the category corresponding to the sampling vector is close to the category of the second high-dimensional vector in the high-dimensional vector corresponding to the data to be processed, the distance between the target sampling point and the second high-dimensional vector is the shortest.
[0150] In some embodiments, in response to the number of categories of the data to be processed being 1, in the vector space, the at least one sampling vector is located within a sphere having the high-dimensional vector as the center and a preset distance as the radius;
[0151] In response to the number of categories of the data to be processed being 2, in the vector space, the at least one sampling vector is located in the space between the two high-dimensional vectors;
[0152] In response to the number of categories of the data to be processed being greater than or equal to 3, in the vector space, the at least one sampling vector is located in the space between the high-dimensional vectors corresponding to the data to be processed.
[0153] The vector decoding unit 704 is specifically configured to input the at least one sampling vector into a vector decoder, and confirm that the output of the vector decoder is a sample of the data to be processed.
[0154] In some embodiments, the sample determination device 700 further includes a filtering unit 705 .
[0155] The filtering unit 705 is configured to, after confirming that the sample data corresponding to the at least one sampling vector is a sample of the data to be processed, filter the sample of the data to be processed based on a first condition and confirm that the filtered sample is a target sample of the data to be processed;
[0156] The first condition includes at least one of the same category, the same expression and semantic coherence.
[0157] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.
[0158] Figure 9 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0159] like Figure 9 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0160] Multiple components in the electronic device 800 are connected to the I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.
[0161] The computing unit 801 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the sample determination method. For example, in some embodiments, the sample determination method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the sample determination method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the sample determination method by any other appropriate means (e.g., by means of firmware).
[0162] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0163] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.
[0164] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0165] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0166] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.
[0167] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.
[0168] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.
[0169] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means two or more, unless otherwise specifically defined.
[0170] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.
Claims
1. A sample determination method, the method comprising: Determine the high-dimensional vector corresponding to the data to be processed; Determining a target sampling point based on the high-dimensional vector and the number of categories of the high-dimensional vector; Based on the target sampling point, determining at least one sampling vector in a vector space; Confirm that the sampling data corresponding to the at least one sampling vector is a sample of the data to be processed.
2. According to the method of claim 1, the determining of the vector corresponding to the data to be processed is implemented by a vector encoder, and the method further comprises: Training the vector encoder based on the data training set so that the distance between high-dimensional vectors corresponding to the training data of the same category in the vector space is less than a first threshold, and the distance between high-dimensional vectors corresponding to the training data of different categories in the vector space is greater than a second threshold; Determining the high-dimensional vector corresponding to the data to be processed includes: The data to be processed is input into a vector encoder, and it is confirmed that the output of the vector encoder is a high-dimensional vector corresponding to the data to be processed.
3. The method according to claim 1, wherein the determining the target sampling point based on the high-dimensional vector and the number of categories of the high-dimensional vector comprises one of the following: In response to the number of categories of the data to be processed being 1, the target sampling point is the location of the high-dimensional vector; In response to the number of categories of the data to be processed being 2, the target sampling point is on a connecting line corresponding to two high-dimensional vectors; in, If the target sampling point is on the connecting line corresponding to two high-dimensional vectors, and the category corresponding to the sampling vector is close to the category of the first high-dimensional vector of the two high-dimensional vectors, then the distance between the target sampling point and the first high-dimensional vector is the shortest.
4. The method according to claim 1, wherein determining the target sampling point based on the high-dimensional vector corresponding to the data to be processed comprises: In response to the number of categories of the data to be processed being greater than or equal to 3, the target sampling point is in a region enclosed by the high-dimensional vector corresponding to the data to be processed; If the category corresponding to the sampling vector is close to the category of the second high-dimensional vector in the high-dimensional vector corresponding to the data to be processed, the distance between the target sampling point and the second high-dimensional vector is the shortest.
5. The method according to claim 3 or 4, In response to the number of categories of the data to be processed being 1, in the vector space, the at least one sampling vector is located within a sphere with the high-dimensional vector as the center and a preset distance as the radius; In response to the number of categories of the data to be processed being 2, in the vector space, the at least one sampling vector is located in the space between the two high-dimensional vectors; In response to the number of categories of the data to be processed being greater than or equal to 3, in the vector space, the at least one sampling vector is located in the space between the high-dimensional vectors corresponding to the data to be processed.
6. The method according to claim 1, wherein the step of confirming that the sampled data corresponding to the at least one sampling vector is a sample of the data to be processed comprises: The at least one sampling vector is input to a vector decoder, and an output of the vector decoder is confirmed to be a sample of the data to be processed.
7. The method according to claim 1 or 6, after confirming that the sampling data corresponding to the at least one sampling vector is a sample of the data to be processed, the method further comprises: Filtering samples of the data to be processed based on the first condition, and confirming that the filtered samples are target samples of the data to be processed; The first condition includes at least one of the same category, the same expression and semantic coherence.
8. A sample determination device, the device comprising: A vector encoding unit, used to determine a high-dimensional vector corresponding to the data to be processed; a target sampling point confirmation unit, configured to determine a target sampling point based on the high-dimensional vector and the number of categories of the high-dimensional vector; A sampling unit, configured to determine at least one sampling vector in a vector space based on the target sampling point; The vector decoding unit is used to confirm that the sampling data corresponding to the at least one sampling vector is a sample of the data to be processed.
9. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.