Adaptive difficult negative sample sampling method, device, equipment and storage medium

Abstract

The present application relates to the field of deep learning technology, and in particular to an adaptive difficult negative sample sampling method, apparatus, equipment and storage medium, which realizes that in a batch of text data, when the feature vector of each sample changes, the Alignment value and the Uniformity value are determined according to the similarity between samples. By calculating the Alignment value and the Uniformity value, the number of difficult negative samples sampled in each batch of text data is adaptively determined. The technical problem in the prior art that the number of difficult negative samples cannot be adaptively determined based on the samples in each batch of text data is solved. After finding the difficult negative samples, each batch of text data can be better processed through the processing of the text model, and the text matching effect can be improved. It can be used for intelligent question-and-answer applications before or after the sale of financial products. By matching the corresponding standard questions in the question-and-answer library with the user's questions, the standard answers corresponding to the standard questions are sent to the user.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to an adaptive difficult negative sample sampling method, apparatus, device and storage medium. Background Art

[0002] The contrastive learning process involves sampling negative samples, which directly impacts the processing performance of each batch of text data. Sampling difficult negative examples is particularly critical during negative sampling, as these difficult negative examples are extremely difficult for the text model to distinguish from positive examples. During contrastive learning training, a common negative sampling method uses samples within a training batch with different labels than the current sample as negative examples. The latest method for sampling difficult negative examples uses samples within a training batch with the highest similarity to the current sample and a different label as difficult negative examples. However, this method still has flaws. At the initialization of the text model, the text model parameters are not fully trained, so calculating the similarity between two samples using the features output by the current text model is inaccurate. However, if all samples within a training batch with different labels than the current sample are considered difficult negative examples, some simple negative examples within the batch will also be considered difficult negative examples. However, simple negative examples do not need to be reused for training, making it difficult to identify truly difficult negative examples for a batch of text data. After finding difficult negative samples, each batch of text data can be better processed. It can be used for intelligent question-and-answer applications before or after the sale of financial products. By matching user questions with corresponding standard questions in the question-and-answer library, the standard answers corresponding to the standard questions are sent to the user. Summary of the Invention

[0003] The main purpose of this application is to provide an adaptive difficult negative sample sampling method, device, equipment and storage medium, aiming to solve the technical problem in the prior art that the number of difficult negative samples cannot be adaptively determined based on the samples in each batch of text data.

[0004] In order to achieve the above-mentioned object of the invention, the present application proposes an adaptive difficult negative sample sampling method, which includes:

[0005] Obtaining a batch of text data, wherein the text data includes positive sample pairs, negative sample pairs, and the number of samples;

[0006] Calculating the similarity of the positive sample pairs, and calculating the average of the similarities of the positive sample pairs to obtain an Alignment value;

[0007] Calculate the similarity of all sample pairs, input the similarity of the sample pairs into a preset exponential function, and calculate the average of the output values ​​of the preset exponential function to obtain a uniformity value;

[0008] Calculate the number of difficult negative samples in the corresponding batch according to the Alignment value, the Uniformity value, and the number of samples in the batch;

[0009] The number of difficult negative samples in the batch of text data is determined according to the number of difficult negative samples in the corresponding batch and the number of samples.

[0010] Furthermore, the calculating the similarity of the positive sample pairs and calculating the average of the similarities of the positive sample pairs to obtain an Alignment value includes:

[0011] Calculate using the following formula:

[0012]

[0013] Among them, t and v are the feature vectors of each sample in the positive sample pair, sim is the cosine similarity of the positive sample pair, B is a batch of samples, γ align is the Alignment value, and E is the expectation.

[0014] Furthermore, the similarity of all sample pairs is calculated, the similarity of the sample pairs is input into a preset exponential function, and the average of the output values ​​of the preset exponential function is calculated to obtain the Uniformity value, including:

[0015] Calculate using the following formula:

[0016]

[0017] Among them, t and v are the feature vectors of the samples, sim is the cosine similarity of the samples, e is the preset exponential function, B is a batch of samples, γ uniform is the Uniformity value, and E is the expectation.

[0018] Furthermore, the calculating the number of difficult negative samples in the corresponding batch according to the Alignment value, the Uniformity value, and the number of samples in the batch includes:

[0019] Calculate using the following formula:

[0020] K′=B|*cos(γ align +γ uniform )

[0021] Among them, γ align is the Alignment value, γ uniform is the Uniformity value, |B| is the number of samples in a batch.

[0022] Furthermore, determining the number of difficult negative samples in the batch of text data according to the number of difficult negative samples in the corresponding batch and the number of samples includes:

[0023] Calculate using the following formula:

[0024] K=min(K′,B|-1)&max(K′,1)

[0025] Among them, K′ is the number of difficult negative samples in the corresponding batch, and |B| is the number of samples in a batch.

[0026] Furthermore, the adaptive difficult negative sample sampling device comprises:

[0027] A sample acquisition module is used to acquire a batch of text data, wherein the text data includes positive sample pairs, negative sample pairs and sample quantity;

[0028] A first calculation module is used to calculate the similarity of the positive sample pairs and calculate the average value of the similarity of the positive sample pairs to obtain an alignment value;

[0029] A second calculation module is used to calculate the similarity of all sample pairs, input the similarity of the sample pairs into a preset exponential function, and calculate the average of the output values ​​of the preset exponential function to obtain a uniformity value;

[0030] A third calculation module is used to calculate the number of difficult negative samples in the corresponding batch according to the Alignment value, the Uniformity value and the number of samples in the batch;

[0031] A fourth calculation module is used to determine the number of difficult negative samples in the batch of text data according to the number of difficult negative samples in the corresponding batch and the number of samples.

[0032] Furthermore, the adaptive difficult negative sample sampling device comprises:

[0033] The Alignment value calculation submodule is used to calculate using the following formula:

[0034]

[0035] Among them, t and v are the feature vectors of each sample in the positive sample pair, sim is the cosine similarity of the positive sample pair, B is a batch of samples, γ align is the Alignment value, and E is the expectation.

[0036] Furthermore, the adaptive difficult negative sample sampling device comprises:

[0037] The Uniformity value calculation submodule is used to calculate using the following formula:

[0038]

[0039] Among them, t and v are the feature vectors of the samples, sim is the cosine similarity of the samples, e is the preset exponential function, B is a batch of samples, γ uniform is the Uniformity value, and E is the expectation.

[0040] The present application also provides a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and wherein the processor implements the steps of any of the above methods when executing the computer program.

[0041] The present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the above-described methods when executed by a processor.

[0042] The present application provides an adaptive difficult negative sample sampling method, which realizes that in a batch of text data, when the feature vector of each sample changes, the Alignment value and the Uniformity value are determined according to the similarity between samples. By calculating the Alignment value and the Uniformity value, the number of difficult negative samples sampled in each batch of text data is adaptively determined. The technical problem in the prior art that the number of difficult negative samples cannot be adaptively determined based on the text model samples is solved, thereby improving the effect of text model training. After finding the difficult negative samples, each batch of text data can be better processed through the processing of the text model, and the text matching effect can be improved. It can be used for intelligent question-and-answer applications before or after the sale of financial products. By matching the corresponding standard questions in the question-and-answer library with the user's questions, the standard answers corresponding to the standard questions are sent to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Schematic diagram of the process of an adaptive difficult negative sample sampling method according to an embodiment of the present application;

[0044] Figure 2 This is a schematic block diagram of the structure of an adaptive difficult negative sample sampling device according to an embodiment of the present application;

[0045] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present application.

[0046] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0048] Reference Figure 1 In an embodiment of the present application, an adaptive difficult negative sample sampling method is provided, the method comprising:

[0049] S1. Obtain a batch of text data, wherein the text data includes positive sample pairs, negative sample pairs, and sample quantity;

[0050] S2. Calculate the similarity of the positive sample pairs, and calculate the average value of the similarity of the positive sample pairs to obtain an Alignment value;

[0051] S3. Calculate the similarity of all sample pairs, input the similarity of the sample pairs into a preset exponential function, and calculate the average of the output values ​​of the preset exponential function to obtain a Uniformity value;

[0052] S4. Calculate the number of difficult negative samples in the corresponding batch according to the Alignment value, the Uniformity value, and the number of samples in the batch;

[0053] S5. Determine the number of difficult negative samples in the batch of text data according to the number of difficult negative samples in the corresponding batch and the number of samples.

[0054] In the above embodiment, it is achieved that in a batch of text data, when the feature vector of each sample changes, the Alignment value and the Uniformity value are determined according to the similarity between samples. By calculating the Alignment value and the Uniformity value, the number of difficult negative samples of each batch of text data is adaptively determined. The technical problem in the prior art that the number of difficult negative samples cannot be adaptively determined based on the samples in each batch of text data is solved. After finding the difficult negative samples, each batch of text data can be better processed through the processing of the text model, and the text matching effect can be improved. It can be used for intelligent question-and-answer applications before or after the sale of financial products. By matching the corresponding standard questions in the question-and-answer library with the user's questions, the standard answers corresponding to the standard questions are sent to the user.

[0055] As described in step S1 above, a batch of text data is obtained, where the text data includes positive sample pairs, negative sample pairs, and the number of samples.

[0056] Specifically, for step S1, when the intelligent question-answering application is used, the user's question is matched with the corresponding standard question in the question-answer library, and the standard answer corresponding to the standard question is sent to the user. In order to improve the matching effect, a large number of positive sample pairs and negative sample pairs are required to train the text model. To solve the problem of insufficient labeled data, contrastive learning provides such an idea: construct similar sample pairs (i.e., positive sample pairs) and dissimilar sample pairs (i.e., negative sample pairs) respectively, and achieve the purpose of training the text model by distinguishing between positive sample pairs and negative sample pairs. During training, the text model will shorten the distance between positive sample pairs and increase the distance between negative sample pairs. In text model training, there will be some negative samples that are difficult to distinguish in a large amount of training data, which are called difficult negative samples. Find such negative samples and then conduct targeted training. After finding the difficult negative samples, each batch of text data can be better processed through the processing of the text model to improve the text matching effect. Simple negative samples do not need to be reused for training. Therefore, the number of difficult negative samples needs to be determined based on the text data of each batch. After finding the difficult negative samples, each batch of text data can be better processed through text model processing to improve the text matching effect. It can be used for intelligent question-and-answer applications before or after the sale of financial products. By matching the corresponding standard questions in the question-and-answer library with the user's questions, the standard answers corresponding to the standard questions can be sent to the user.

[0057] As in step S2 above, the similarity of the positive sample pairs is calculated, and the average value of the similarity of the positive sample pairs is calculated to obtain the Alignment value.

[0058] Specifically, in step S2, the Alignment value represents the distance between samples within a positive pair. A smaller Alignment value indicates closer vectors between samples within the positive pair, more similar samples within the positive pair, and better comparative learning results. The Alignment value, representing the distance between samples within a positive pair, serves as one of the factors influencing the number of difficult negative samples in the current batch of text data.

[0059] As in step S3 above, the similarities of all sample pairs are calculated, the similarities of the sample pairs are input into a preset exponential function, and the average value of the output values ​​of the preset exponential function is calculated to obtain the Uniformity value.

[0060] Specifically, for step S3, the Uniformity value represents the uniformity of the distribution of all samples in a batch. The smaller the uniformity value, the more uniform the vector distribution, the more uniform the sample distribution, the higher the information entropy, and the better the effect of contrastive learning. The Uniformity value represents the uniformity of the distribution of all samples in a batch through a specific numerical value, which serves as one of the influencing factors for determining the number of difficult negative samples in the current batch of text data.

[0061] As in steps S4 and S5 above, the number of difficult negative samples in the corresponding batch is calculated based on the Alignment value, the Uniformity value and the number of samples in the batch; and the number of difficult negative samples in the text data of the batch is determined based on the number of difficult negative samples in the corresponding batch and the number of samples.

[0062] Specifically, for steps S4 and S5, in each batch of text data, there will be some difficult-to-distinguish negative samples in a large amount of data, namely difficult negative samples. Finding such negative samples and then conducting targeted training can better process each batch of text data and improve the text matching effect. Since the actual number of samples may be very large, the number of training samples is divided into batches. First, based on the sample situation of a batch, the number of difficult negative samples in the corresponding batch is determined. Then, based on the number of difficult negative samples in the corresponding batch, the number of difficult negative samples in the current batch of text data is determined.

[0063] An embodiment of the present application provides an adaptive difficult negative sample sampling method, the method comprising:

[0064] S201, calculate using the following formula:

[0065]

[0066] Among them, t and v are the feature vectors of each sample in the positive sample pair, sim is the cosine similarity of the positive sample pair, B is a batch of samples, γ align is the Alignment value, and E is the expectation.

[0067] In the above embodiment, the alignment value is determined by the relationship between the positive sample pairs in a batch.

[0068] As described in step S201 above, the following formula is used for calculation:

[0069]

[0070] Among them, t and v are the feature vectors of each sample in the positive sample pair, sim is the cosine similarity of the positive sample pair, B is a batch of samples, γalign is the Alignment value, and E is the expectation.

[0071] Specifically, for step S201, for all positive sample pairs t and v in a batch (B), calculate their respective cosine similarities sim, and calculate the average of the similarities of all positive sample pairs. align Is a decimal number between 0 and 1.

[0072] An embodiment of the present application provides an adaptive difficult negative sample sampling method, the method comprising:

[0073] S301, calculate using the following formula:

[0074]

[0075] Among them, t and v are the feature vectors of the samples, sim is the cosine similarity of the samples, e is the preset exponential function, B is a batch of samples, γ uniform is the Uniformity value, and E is the expectation.

[0076] In the above embodiment, the Uniformity value is determined by the relationship between all sample vectors in a batch.

[0077] As described in step S301 above, the following formula is used for calculation:

[0078]

[0079] Among them, t and v are the feature vectors of the samples, sim is the cosine similarity of the samples, e is the preset exponential function, B is a batch of samples, γ uniform is the Uniformity value, and E is the expectation.

[0080] Specifically, for step S301, for all samples t and v in a batch (B), calculate their respective cosine similarities sim. No distinction is made between positive and negative sample pairs here, that is, the similarity between the first sample and the first sample and all other samples in the current batch is calculated separately, and so on for other samples. Then input the obtained similarity into the exponential function e, calculate the average of all similarities, and then perform a log operation. uniform It is also a decimal between 0 and 1.

[0081] An embodiment of the present application provides an adaptive difficult negative sample sampling method, the method comprising:

[0082] S401, calculate using the following formula:

[0083] K′=|B|*cos(γalign +γ uniform )

[0084] Among them, γ align is the Alignment value, γ uniform is the Uniformity value, |B| is the number of samples in a batch.

[0085] In the above embodiment, the number of difficult negative samples in the corresponding batch can be determined.

[0086] As described in step S401 above, the following formula is used for calculation:

[0087] K′=|B|*cos(γ align +γ uniform )

[0088] Among them, γ align is the Alignment value, γ uniform is the Uniformity value, |B| is the number of samples in a batch.

[0089] Specifically, for step S401, the Alignment value and the Uniformity value obtained in the above step are added together, then calculated using the cosine function cos, and finally multiplied by the number of samples in the batch to calculate the number K′ of difficult negative samples to be selected. The value of K′ is the value obtained by rounding down the result of the above formula.

[0090] An embodiment of the present application provides an adaptive difficult negative sample sampling method, the method comprising:

[0091] S501, calculate using the following formula:

[0092] K=min(K′,|B|-1)&max(K′,1)

[0093] Where K′ is the number of difficult negative samples in the corresponding batch, and |B| is the number of samples in a batch. As described in step S501 above, the following formula is used for calculation:

[0094] K=min(K′,|B|-1)&max(K′,1)

[0095] Among them, K′ is the number of difficult negative samples in the corresponding batch, and |B| is the number of samples in a batch.

[0096] Specifically, for step S501, this step can be used to determine the number of difficult negative samples in the current batch of text data. In actual situations, positive samples are usually determined in advance and are difficult to expand and modify, but negative samples have a very large selection space. According to the above formula, the maximum number of negative samples is |B|-1, and the minimum number of negative samples is 1. In general, during comparative learning training, there will be some negative samples that are difficult to distinguish in a large amount of training data, which are difficult negative samples. The commonly used negative sample sampling method is to use samples with different labels from the current sample in a training batch as negative samples. The latest method for sampling difficult negative samples is to calculate the samples with the highest similarity to the current sample and different labels in a training batch as difficult negative samples. However, this method still has defects. The reason is that when the text model is initialized, the text model parameters are not fully trained, so it is inaccurate to calculate the similarity of the two samples by outputting the features of the two samples by the current text model. However, if all samples in a training batch with different labels than the current sample are treated as hard negative samples, some simple negative samples within the batch will also be treated as hard negative samples. However, simple negative samples do not need to be reused for training, making it difficult to find truly hard negative samples within a batch of text data. Finding hard negative samples and then conducting targeted training can better process each batch of text data and improve text matching results. This step enables adaptive determination of the number of hard negative samples in each batch of text data. This solves the technical problem in the prior art of not being able to adaptively determine the number of hard negative samples based on the samples in each batch of text data.

[0097] Reference Figure 2 , is a structural block diagram of an adaptive difficult negative sample sampling device in one embodiment of the present application, the device comprising:

[0098] The sample acquisition module 100 is used to acquire a batch of text data, wherein the text data includes positive sample pairs, negative sample pairs and the number of samples;

[0099] A first calculation module 200 is used to calculate the similarity of the positive sample pairs and calculate the average value of the similarity of the positive sample pairs to obtain an alignment value;

[0100] The second calculation module 300 is used to calculate the similarity of all sample pairs, input the similarity of the sample pairs into a preset exponential function, and calculate the average of the output values ​​of the preset exponential function to obtain a uniformity value;

[0101] A third calculation module 400 is configured to calculate the number of difficult negative samples in a corresponding batch according to the Alignment value, the Uniformity value, and the number of samples in the batch;

[0102] The fourth calculation module 500 is configured to determine the number of difficult negative samples in the batch of text data according to the number of difficult negative samples in the corresponding batch and the number of samples.

[0103] Specifically, for the sample acquisition module 100, when the intelligent question and answer application is used, the standard answers corresponding to the standard questions are sent to the user by matching the user's questions with the corresponding standard questions in the question and answer library. In order to improve the matching effect, a large number of positive sample pairs and negative sample pairs are required to train the text model. In order to solve the problem of insufficient labeled data, contrastive learning provides such an idea: construct similar sample pairs (i.e., positive sample pairs) and dissimilar sample pairs (i.e., negative sample pairs) respectively, and achieve the purpose of training the text model by distinguishing between positive sample pairs and negative sample pairs. During training, the text model will shorten the distance between positive sample pairs and increase the distance between negative sample pairs. In text model training, there will be some negative samples that are difficult to distinguish in a large amount of training data, which are called difficult negative samples. Find such negative samples and then conduct targeted training. After finding the difficult negative samples, each batch of text data can be better processed through the processing of the text model to improve the text matching effect. Simple negative samples do not need to be reused for training. Therefore, the number of difficult negative samples needs to be determined based on the text data of each batch. After finding the difficult negative samples, each batch of text data can be better processed through text model processing to improve the text matching effect. It can be used for intelligent question-and-answer applications before or after the sale of financial products. By matching the corresponding standard questions in the question-and-answer library with the user's questions, the standard answers corresponding to the standard questions can be sent to the user.

[0104] Specifically, for the first calculation module 200, the alignment value represents the distance between samples within a positive sample pair. The smaller the alignment value, the closer the vectors between samples within the positive sample pair are, the more similar the samples within the positive sample pair are, and the better the comparative learning effect. The alignment value represents the distance between samples within a positive sample pair through a specific numerical value, and serves as one of the factors affecting the number of difficult negative samples in the current batch of text data training.

[0105] Specifically, for the second computing module 300, the Uniformity value represents the uniformity of the distribution of all samples in a batch. The smaller the uniformity value, the more uniform the vector distribution, the more uniform the sample distribution, the higher the information entropy, and the better the effect of comparative learning. The Uniformity value represents the uniformity of the distribution of all samples in a batch through a specific numerical value, which serves as one of the influencing factors for determining the number of difficult negative samples in the training process of the text data of the current batch.

[0106] Specifically, for the third calculation module 400 and the fourth calculation module 500, in each batch of text data, there will be some difficult-to-distinguish negative samples in a large amount of data, which are called difficult negative samples. Finding such negative samples and then conducting targeted training can better process each batch of text data and improve the text matching effect. Since the actual number of samples may be large, the number of training samples is divided into batches. First, based on the sample situation of a batch, the number of difficult negative samples in the corresponding batch is determined. Then, based on the number of difficult negative samples in the corresponding batch, the number of difficult negative samples in the current batch of text data is determined.

[0107] Reference Figure 2 In one embodiment, the above-mentioned adaptive difficult negative sample sampling device further includes:

[0108] The Alignment value calculation submodule is used to calculate using the following formula:

[0109]

[0110] Among them, t and v are the feature vectors of each sample in the positive sample pair, sim is the cosine similarity of the positive sample pair, B is a batch of samples, γ align is the Alignment value, and E is the expectation.

[0111] Specifically, for the Alignment value calculation submodule, for all positive sample pairs t and v in a batch (B), calculate their respective cosine similarities sim, and calculate the average similarity of all positive sample pairs. The calculated γ align Is a decimal number between 0 and 1.

[0112] Reference Figure 2 In one embodiment, the above-mentioned adaptive difficult negative sample sampling device further includes:

[0113] The Uniformity value calculation submodule is used to calculate using the following formula:

[0114]

[0115] Among them, t and v are the feature vectors of the samples, sim is the cosine similarity of the samples, e is the preset exponential function, B is a batch of samples, γ uniform is the Uniformity value, and E is the expectation.

[0116] Specifically, for the Uniformity value calculation submodule, for all samples t and v in a batch (B), the cosine similarity sim is calculated for each. No distinction is made between positive and negative sample pairs here, that is, the similarity of the first sample with the first sample and all other samples in the current batch is calculated separately, and so on for other samples. The obtained similarity is then input into the exponential function e, and the average of all similarities is calculated, and then the log operation is performed. uniform It is also a decimal between 0 and 1.

[0117] Reference Figure 2 In one embodiment, the above-mentioned adaptive difficult negative sample sampling device further includes:

[0118] The module for calculating the number of difficult negative samples in a batch is used to calculate using the following formula:

[0119] K′=|B|*cos(γ align +γ uniform )

[0120] Among them, γ align is the Alignment value, γ uniform is the Uniformity value, |B| is the number of samples in a batch.

[0121] Specifically, for the batch difficult negative sample number calculation module, the Alignment value and Uniformity value obtained in the above steps are added together, then calculated through the cosine function cos, and finally multiplied by the number of samples in the batch to calculate the number of difficult negative samples to be selected K'. The value of K' is the value after rounding down the result of the above formula.

[0122] Reference Figure 2 In one embodiment, the above-mentioned adaptive difficult negative sample sampling device further includes:

[0123] The module for calculating the number of difficult negative samples is used to calculate using the following formula:

[0124] K=min(K′,B|-1)&max(K′,1)

[0125] Among them, K′ is the number of difficult negative samples in the corresponding batch, and |B| is the number of samples in a batch.

[0126] Specifically, the module for calculating the number of difficult negative samples determines the number of difficult negative samples in the current batch of text data. In practice, positive samples are usually predetermined and difficult to expand or modify, but negative samples offer a wide range of choices. According to the above formula, the maximum number of negative samples is |B|-1, and the minimum number is 1. During contrastive learning training, a large amount of training data often contains some difficult-to-distinguish negative samples, known as difficult negative samples. A common negative sample sampling method uses samples within a training batch that have different labels from the current sample as negative samples. The latest method for sampling difficult negative samples is to calculate the samples within a training batch that have the highest similarity to the current sample and have different labels as difficult negative samples. However, this method still has flaws. The reason is that when the text model is initially initialized, the text model parameters are not fully trained. Therefore, calculating the similarity of two samples using the features output by the current text model is inaccurate. However, if all samples in a training batch with different labels than the current sample are treated as hard negative samples, some simple negative samples within the batch will also be treated as hard negative samples. However, simple negative samples do not need to be reused for training, making it difficult to find truly hard negative samples within a batch of text data. Finding hard negative samples and then conducting targeted training can better process each batch of text data and improve text matching results. This step enables adaptive determination of the number of hard negative samples in each batch of text data. This solves the technical problem in the prior art of not being able to adaptively determine the number of hard negative samples based on the samples in each batch of text data.

[0127] Reference Figure 3 In the embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer design is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store operating data of the adaptive difficult negative sample sampling method, etc. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an adaptive difficult negative sample sampling method of any of the above embodiments is implemented.

[0128] In one embodiment, the above-mentioned adaptive difficult negative sample sampling method includes:

[0129] Obtaining a batch of text data, wherein the text data includes positive sample pairs, negative sample pairs, and the number of samples;

[0130] Calculating the similarity of the positive sample pairs, and calculating the average of the similarities of the positive sample pairs to obtain an Alignment value;

[0131] Calculate the similarity of all sample pairs, input the similarity of the sample pairs into a preset exponential function, and calculate the average of the output values ​​of the preset exponential function to obtain a uniformity value;

[0132] Calculate the number of difficult negative samples in the corresponding batch according to the Alignment value, the Uniformity value, and the number of samples in the batch;

[0133] The number of difficult negative samples in the batch of text data is determined according to the number of difficult negative samples in the corresponding batch and the number of samples.

[0134] In one embodiment, calculating the similarity of the positive sample pairs and calculating the average of the similarities of the positive sample pairs to obtain an alignment value includes:

[0135] Calculate using the following formula:

[0136]

[0137] Among them, t and v are the feature vectors of each sample in the positive sample pair, sim is the cosine similarity of the positive sample pair, B is a batch of samples, γ align is the Alignment value, and E is the expectation.

[0138] In one embodiment, calculating the similarity of all sample pairs, inputting the similarity of the sample pairs into a preset exponential function, and calculating the average of the output values ​​of the preset exponential function to obtain the Uniformity value includes:

[0139] Calculate using the following formula:

[0140]

[0141] Among them, t and v are the feature vectors of the samples, sim is the cosine similarity of the samples, e is the preset exponential function, B is a batch of samples, γ uniform is the Uniformity value, and E is the expectation.

[0142] In one embodiment, calculating the number of difficult negative samples in the corresponding batch according to the alignment value, the uniformity value, and the number of samples in the batch includes:

[0143] Calculate using the following formula:

[0144] K′=|B|*cos(γ align +γ uniform )

[0145] Among them, γ align is the Alignment value, γ uniform is the Uniformity value, |B| is the number of samples in a batch.

[0146] In one embodiment, determining the number of difficult negative samples in the batch of text data based on the number of difficult negative samples in the corresponding batch and the sample size includes:

[0147] Calculate using the following formula:

[0148] K=min(K′,|B|-1)&max(K′,1)

[0149] Among them, K′ is the number of difficult negative samples in the corresponding batch, and |B| is the number of samples in a batch.

[0150] Those skilled in the art will understand that Figure 3 The structure shown in is merely a block diagram of a portion of the structure related to the present application solution and does not constitute a limitation on the computer device to which the present application solution is applied.

[0151] One embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements an adaptive difficult negative sample sampling method. It is understood that the computer-readable storage medium in this embodiment can be either a volatile readable storage medium or a non-volatile readable storage medium.

[0152] In one embodiment, the above-mentioned adaptive difficult negative sample sampling method includes:

[0153] Obtaining a batch of text data, wherein the text data includes positive sample pairs, negative sample pairs, and the number of samples;

[0154] Calculating the similarity of the positive sample pairs, and calculating the average of the similarities of the positive sample pairs to obtain an Alignment value;

[0155] Calculate the similarity of all sample pairs, input the similarity of the sample pairs into a preset exponential function, and calculate the average of the output values ​​of the preset exponential function to obtain a uniformity value;

[0156] Calculate the number of difficult negative samples in the corresponding batch according to the Alignment value, the Uniformity value, and the number of samples in the batch;

[0157] The number of difficult negative samples in the batch of text data is determined according to the number of difficult negative samples in the corresponding batch and the number of samples.

[0158] In one embodiment, calculating the similarity of the positive sample pairs and calculating the average of the similarities of the positive sample pairs to obtain an alignment value includes:

[0159] Calculate using the following formula:

[0160]

[0161] Among them, t and v are the feature vectors of each sample in the positive sample pair, sim is the cosine similarity of the positive sample pair, B is a batch of samples, γ align is the Alignment value, and E is the expectation.

[0162] In one embodiment, calculating the similarity of all sample pairs, inputting the similarity of the sample pairs into a preset exponential function, and calculating the average of the output values ​​of the preset exponential function to obtain the Uniformity value includes:

[0163] Calculate using the following formula:

[0164]

[0165] Among them, t and v are the feature vectors of the samples, sim is the cosine similarity of the samples, e is the preset exponential function, B is a batch of samples, γ uniform is the Uniformity value, and E is the expectation.

[0166] In one embodiment, calculating the number of difficult negative samples in the corresponding batch according to the alignment value, the uniformity value, and the number of samples in the batch includes:

[0167] Calculate using the following formula:

[0168] K′=|B|*cos(γ align +γ uniform )

[0169] Among them, γ align is the Alignment value, γ uniform is the Uniformity value, |B| is the number of samples in a batch.

[0170] In one embodiment, determining the number of difficult negative samples in the batch of text data based on the number of difficult negative samples in the corresponding batch and the sample size includes:

[0171] Calculate using the following formula:

[0172] K=min(K′,|B|-1)&max(K′,1)

[0173] Among them, K′ is the number of difficult negative samples in the corresponding batch, and |B| is the number of samples in a batch.

[0174] The present application provides an adaptive difficult negative sample sampling method, which realizes that in a batch of text data, when the feature vector of each sample changes, the Alignment value and the Uniformity value are determined according to the similarity between samples. By calculating the Alignment value and the Uniformity value, the number of difficult negative samples sampled in each batch of text data is adaptively determined. This solves the technical problem in the prior art that the number of difficult negative samples cannot be adaptively determined based on the samples in each batch of text data. After finding the difficult negative samples, each batch of text data can be better processed through the processing of the text model, and the text matching effect can be improved. It can be used for intelligent question-and-answer applications before or after the sale of financial products. By matching the corresponding standard questions in the question-and-answer library with the user's questions, the standard answers corresponding to the standard questions are sent to the user.

[0175] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAM bus dynamic RAM (RDRAM), etc.

[0176] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0177] The above description is only a preferred embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An adaptive difficult negative sample sampling method, characterized in that: The method comprises: Obtaining a batch of text data, wherein the text data includes positive sample pairs, negative sample pairs, and the number of samples; Calculating the similarity of the positive sample pairs, and calculating the average of the similarities of the positive sample pairs to obtain an Alignment value; Calculate the similarity of all sample pairs, input the similarity of the sample pairs into a preset exponential function, and calculate the average of the output values ​​of the preset exponential function to obtain a uniformity value; Calculate the number of difficult negative samples in the corresponding batch according to the Alignment value, the Uniformity value, and the number of samples in the batch; Determining the number of difficult negative samples in the batch of text data according to the number of difficult negative samples in the corresponding batch and the number of samples; Calculating the similarity of the positive sample pairs and calculating the average of the similarities of the positive sample pairs to obtain an alignment value includes: Calculate using the following formula: Among them, t and v are the feature vectors of each sample in the positive sample pair, sim is the cosine similarity of the positive sample pair, B is a batch of samples, is the Alignment value, E is the expectation; The calculating of the similarity of all sample pairs, inputting the similarity of the sample pairs into a preset exponential function, and calculating the average of the output values ​​of the preset exponential function to obtain the Uniformity value includes: Calculate using the following formula: Among them, sim is the cosine similarity of the sample, e is the preset exponential function, B is a batch of samples, is the Uniformity value, E is the expectation; Calculating the number of difficult negative samples in the corresponding batch according to the Alignment value, the Uniformity value, and the number of samples in the batch includes: Calculate using the following formula: in, is the Alignment value, is the Uniformity value, |B| is the number of samples in a batch.

2. The adaptive difficult negative sample sampling method according to claim 1, characterized in that Determining the number of difficult negative samples in the batch of text data according to the number of difficult negative samples in the corresponding batch and the number of samples includes: Calculate using the following formula: K=min(K′,|B|-1)&max(K′,1) Among them, K′ is the number of difficult negative samples in the corresponding batch, and |B| is the number of samples in a batch.

3. An adaptive difficult negative sample sampling device, characterized in that: The device comprises: A sample acquisition module is used to acquire a batch of text data, wherein the text data includes positive sample pairs, negative sample pairs and sample quantity; A first calculation module is used to calculate the similarity of the positive sample pairs and calculate the average value of the similarity of the positive sample pairs to obtain an alignment value; A second calculation module is used to calculate the similarity of all sample pairs, input the similarity of the sample pairs into a preset exponential function, and calculate the average of the output values ​​of the preset exponential function to obtain a uniformity value; A third calculation module is used to calculate the number of difficult negative samples in the corresponding batch according to the Alignment value, the Uniformity value and the number of samples in the batch; a fourth calculation module, configured to determine the number of difficult negative samples in the batch of text data according to the number of difficult negative samples in the corresponding batch and the number of samples; The device further comprises: The Alignment value calculation submodule is used to calculate using the following formula: Among them, t and v are the feature vectors of each sample in the positive sample pair, sim is the cosine similarity of the positive sample pair, B is a batch of samples, is the Alignment value, E is the expectation; The Uniformity value calculation submodule is used to calculate using the following formula: Among them, sim is the cosine similarity of the sample, e is the preset exponential function, B is a batch of samples, is the Uniformity value, E is the expectation; The module for calculating the number of difficult negative samples in a batch is used to calculate using the following formula: in, is the Alignment value, is the Uniformity value, |B| is the number of samples in a batch.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 or 2 are implemented.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 1 or 2 are implemented.

Images