Text similarity calculation model generation method, device, equipment and storage medium

Abstract

The present application relates to the field of computer technology, and the present application provides a method, device, equipment and storage medium for generating a text similarity calculation model, wherein the method obtains a first sample category vector corresponding to a sample data set, and performs a normalization operation on the first sample category vector to generate a second sample category vector; obtains a first sentence vector corresponding to a target sentence, and normalizes the sentence vector through a BERT model to generate a second sentence vector; calculates the similarity between the target sentence and the sample data set based on the second sentence vector and the second sample category vector; and generates a text similarity calculation model corresponding to the target sentence based on the similarity. In the above manner, by vectorizing the sample data and the target sentence, and performing a similarity calculation between the normalized sample category vector and the sentence vector, a text similarity calculation model is determined, the reliability of the text similarity calculation model is improved, and the technical problem of low reliability of the current model for calculating text similarity is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment and computer-readable storage medium for generating a text similarity calculation model. Background Art

[0002] With the development of Internet technology, more and more Internet services are using customer service robots to answer a series of questions from users. In the process of answering questions, a large number of questions and answers are often provided to customer service robots as samples through a lot of training. Customer service robots respond to users based on the similarity between the questions raised by users and the samples stored in the database.

[0003] Most of the current sentence similarity algorithms are aimed at sentence pair datasets (i.e., triple datasets of <sentence 1, sentence 2, label>). For text classification datasets, some methods combine sentences of the same type into triples of <sentence 1, sentence 2, 1>, and sentences of different classes into triples of <sentence 1, sentence 2, 0>. However, the number of positive samples composed of data within the same class is far less than the number of negative samples composed of sentences of different classes, and the samples are extremely unbalanced. When downsampling negative samples, the data information of the entire dataset cannot be fully utilized, and the number of positive samples is often far less than the number of negative samples, resulting in low reliability of the model for calculating sample similarity. Therefore, how to solve the low reliability of the model for calculating text similarity has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the present invention is to provide a method, device, equipment and computer-readable storage medium for generating a text similarity calculation model, aiming to solve the technical problem of low reliability of existing models for calculating text similarity.

[0005] To achieve the above-mentioned purpose, the present invention provides a method for generating a text similarity calculation model, the method obtaining a first sample category vector corresponding to a sample data set, and normalizing the first sample category vector to generate a second sample category vector; obtaining a first sentence vector corresponding to a target sentence, and normalizing the sentence vector through a pre-trained language representation BERT model to generate a second sentence vector; calculating the similarity between the target sentence and the sample data set according to the second sentence vector and the second sample category vector; and generating a text similarity calculation model corresponding to the target sentence based on the similarity.

[0006] Further, obtaining a first sample category vector corresponding to the sample data set, and performing a normalization operation on the first sample category vector to generate a second sample category vector includes:

[0007] Setting the first sample category vector based on the number of categories included in the sample data set and the number of samples included in each category;

[0008] The first sample category vector is normalized by L2 norm through a first preset formula to generate the second sample category vector, wherein the first preset formula is as follows:

[0009]

[0010] represents the i-th normalized category vector, C i represents the unnormalized category vector, is the length of the category vector;

[0011] Furthermore, a first sentence vector corresponding to the target sentence is obtained, and the sentence vector is normalized by a pre-trained language representation BERT model to generate a second sentence vector, including:

[0012] Input the target sentence into the BERT model, and use the output result as a common language specification CLS tag;

[0013] Mapping the CLS label to the second sample category vector, and obtaining the first sentence vector based on the mapping result;

[0014] The first sentence vector is normalized by the L2 norm through a second preset formula to generate the second sentence vector, wherein the second preset formula is as follows:

[0015]

[0016] represents the normalized vector of the second sentence, X represents the unnormalized vector of the first sentence, Indicates the length of the sentence vector;

[0017] Further, calculating the similarity between the target sentence and the sample data set according to the second sentence vector and the second sample category vector includes:

[0018] Based on the second sentence vector and the second sample category vector, the similarity is calculated by a third preset formula, wherein the third preset formula is as follows:

[0019]

[0020] The result calculated by the third preset formula is the similarity between the target sentence and the sample data set;

[0021] Furthermore, before generating a text similarity calculation model corresponding to the target sentence based on the similarity, the method includes:

[0022] Through the similarity, a similarity matrix is ​​obtained, wherein the similarity matrix is ​​as follows:

[0023]

[0024] S is the similarity matrix, is the sentence vector, is the sample category vector, S contains n vectors;

[0025] Further, after obtaining the similarity matrix through the similarity, it includes:

[0026] The similarity matrix is ​​encoded with one-hot encoding to obtain the encoded true value, wherein the true value is as follows:

[0027] target = [0,0,1,0,...,0]

[0028] target is a real value array, the number of values ​​in the real value array is the same as the number of vectors in the similarity matrix, only one of them is 1, and the rest are 0;

[0029] Furthermore, generating a text similarity calculation model corresponding to the target sentence based on the similarity includes:

[0030] The loss function between the similarity matrix and the true value array is calculated by a preset loss function formula, and the text similarity calculation model is generated based on the minimum value of the loss function, wherein the preset loss function formula is as follows:

[0031]

[0032] L is the minimization value, and n is the number of vectors in the similarity matrix.

[0033] In addition, to achieve the above-mentioned purpose, the present invention also provides a text similarity calculation model generation device, which includes: a sample category vector generation module, which is used to obtain a first sample category vector corresponding to a sample data set, and normalize the first sample category vector to generate a second sample category vector; a sentence vector generation module, which is used to obtain a first sentence vector corresponding to a target sentence, and normalize the sentence vector through a pre-trained language representation BERT model to generate a second sentence vector; a similarity calculation module, which is used to calculate the similarity between the target sentence and the sample data set according to the second sentence vector and the second sample category vector; and a model generation module, which is used to generate a text similarity calculation model corresponding to the target sentence based on the similarity.

[0034] In addition, to achieve the above-mentioned purpose, the present invention also provides a text similarity calculation model generation device, which includes a processor, a memory, and a text similarity calculation model generation program stored in the memory and executable by the processor, wherein when the text similarity calculation model generation program is executed by the processor, the steps of the text similarity calculation model generation method as described above are implemented.

[0035] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a text similarity calculation model generation program is stored, wherein when the text similarity calculation model generation program is executed by a processor, the steps of the text similarity calculation model generation method as described above are implemented.

[0036] The present invention provides a method for generating a text similarity calculation model, the method obtaining a first sample category vector corresponding to a sample data set, and performing a normalization operation on the first sample category vector to generate a second sample category vector; obtaining a first sentence vector corresponding to a target sentence, and normalizing the sentence vector through a pre-trained language representation BERT model to generate a second sentence vector; calculating the similarity between the target sentence and the sample data set according to the second sentence vector and the second sample category vector; and generating a text similarity calculation model corresponding to the target sentence based on the similarity. In the above manner, by vectorizing the sample data and the target sentence, and performing a similarity calculation between the normalized sample category vector and the sentence vector, a text similarity calculation model is determined, the reliability of the text similarity calculation model is improved, and the technical problem of low reliability of the current model for calculating text similarity is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of the hardware structure of a text similarity calculation model generation device involved in an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of the flow chart of the first embodiment of the method for generating a text similarity calculation model of the present invention;

[0039] Figure 3 A schematic diagram of a flow chart of a second embodiment of a method for generating a text similarity calculation model according to the present invention;

[0040] Figure 4 A schematic diagram of a flow chart of a third embodiment of a method for generating a text similarity calculation model according to the present invention;

[0041] Figure 5 This is a functional module diagram of the first embodiment of the text similarity calculation model generation device of the present invention.

[0042] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0044] The text similarity calculation model generation method involved in the embodiment of the present invention is mainly applied to a text similarity calculation model generation device, which can be a device with display and processing functions such as a PC, a portable computer, or a mobile terminal.

[0045] Reference Figure 1 , Figure 1 The hardware structure diagram of the text similarity calculation model generation device involved in the embodiment of the present invention is shown. In the embodiment of the present invention, the text similarity calculation model generation device may include a processor 1001 (such as a CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface); the memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory, and the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0046] Those skilled in the art will understand that Figure 1 The hardware structure shown in the figure does not constitute a limitation on the text similarity calculation model generation device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0047] Continue to refer to Figure 1 , Figure 1 The memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, and a text similarity calculation model generation program.

[0048] exist Figure 1 In the embodiment, the network communication module is mainly used to connect to the server and perform data communication with the server; and the processor 1001 can call the text similarity calculation model generation program stored in the memory 1005, and execute the text similarity calculation model generation method provided in the embodiment of the present invention.

[0049] An embodiment of the present invention provides a method for generating a text similarity calculation model.

[0050] Reference Figure 2 , Figure 2 It is a flowchart of the first embodiment of the method for generating a text similarity calculation model of the present invention.

[0051] In this embodiment, the text similarity calculation model generation method includes the following steps:

[0052] Step S10, obtaining a first sample category vector corresponding to the sample data set, and performing a normalization operation on the first sample category vector to generate a second sample category vector;

[0053] In this embodiment, there are two forms of normalization methods, one is to convert the number into a decimal between (0, 1), and the other is to convert the dimensional expression into a dimensionless expression. It is mainly proposed for the convenience of data processing. Mapping the data to the range of 0 to 1 for processing is more convenient and fast, and should be classified as a digital signal processing. Normalization is a way to simplify calculations, that is, to transform the dimensional expression into a dimensionless expression and become a scalar.

[0054] There is a data set D, which contains n categories and m samples in each category. Then we set the category vector C = {C1, C2, ..., Cn}, which consists of n vectors, and each vector has l dimensions. The category vector is obtained by random initialization and normalized by L2 norm.

[0055] Step S20, obtaining a first sentence vector corresponding to the target sentence, and normalizing the sentence vector using a pre-trained language representation BERT model to generate a second sentence vector;

[0056] Specifically, the sentence is input into the BERT model (BERT stands for Bidirectional Encoder Representation from Transformers, a pre-trained language representation model), and the vector corresponding to the output result [CLS (Common Language Specification)] label is followed by a mapping layer to map the sentence vector to the same dimension as the category vector, and obtain the sentence vector x. Finally, the L2 norm is used for vector normalization, that is, x = x / |x|.

[0057] The BERT model helps in generating contextual representations for each token. It is even able to capture the context of entire sentences, sentence pairs, or paragraphs.

[0058] BERT basically uses the concept of pre-training a model in an unsupervised manner on a very large dataset for language modeling. Models pre-trained on very large datasets are better able to understand the context of the input sentence. After pre-training, the model can be fine-tuned on a supervised dataset for a specific task to achieve good results.

[0059] [CLS] is a special classification token. The last hidden state of BERT corresponds to this token (h[CLS]) for classification tasks. BERT uses Wordpiece embedding input as a tag.

[0060] In a specific embodiment, the subset of language functions supported by CLS includes common functions of several object-oriented programming languages. Components and tools that comply with CLS can guarantee interoperability with other components and tools that comply with CLS. NET Framework defines CLS as a set of rules that all .NET languages ​​should follow to create applications that can interoperate with other languages. However, it should be noted that in order for languages ​​to interoperate, only the functional objects listed in CLS can be used. These functions are collectively referred to as CLS-compatible functions.

[0061] Step S30, calculating the similarity between the target sentence and the sample data set according to the second sentence vector and the second sample category vector;

[0062] In this embodiment, cosine similarity, also known as cosine similarity, evaluates the similarity of two vectors by calculating the cosine value of the angle between them. Cosine similarity plots the vectors into a vector space, such as the most common two-dimensional space, according to the coordinate values.

[0063] Cosine similarity measures the similarity between two vectors by measuring the cosine of the angle between them. The cosine of a 0-degree angle is 1, while the cosine of any other angle is no greater than 1; and its minimum value is -1. Therefore, the cosine of the angle between two vectors determines whether the two vectors point in roughly the same direction. When two vectors have the same direction, the cosine similarity value is 1; when the angle between the two vectors is 90°, the cosine similarity value is 0; when the two vectors point in completely opposite directions, the cosine similarity value is -1. This result is independent of the length of the vector, but only related to the direction in which the vector points. Cosine similarity is usually used in positive space, so the value given is between -1 and 1.

[0064] Note that these bounds apply to vector spaces of any dimension, and that cosine similarity is most commonly used in high-dimensional positive spaces. For example, in information retrieval, each term is assigned a different dimension, and each dimension is represented by a vector whose values ​​correspond to the frequency of the term in the document. Cosine similarity can therefore give the similarity of two documents in terms of their topics.

[0065] Step S40: generating a text similarity calculation model corresponding to the target sentence based on the similarity.

[0066] In this embodiment, the loss function is mainly used in the training stage of the model. After each batch of training data is sent to the model, the predicted value is output through forward propagation, and then the loss function calculates the difference between the predicted value and the true value, that is, the loss value. After obtaining the loss value, the model updates each parameter through back propagation to reduce the loss between the true value and the predicted value, so that the predicted value generated by the model is closer to the true value, thereby achieving the purpose of learning.

[0067] Since the generated text similarity calculation model has high accuracy and reliability, when the model is used to identify user questions, it can accurately find the corresponding database questions and generate sample answers that can satisfy the users.

[0068] This embodiment provides a method for generating a text similarity calculation model, the method obtaining a first sample category vector corresponding to a sample data set, and performing a normalization operation on the first sample category vector to generate a second sample category vector; obtaining a first sentence vector corresponding to a target sentence, and normalizing the sentence vector through a pre-trained language representation BERT model to generate a second sentence vector; calculating the similarity between the target sentence and the sample data set based on the second sentence vector and the second sample category vector; and generating a text similarity calculation model corresponding to the target sentence based on the similarity. In the above manner, by vectorizing the sample data and the target sentence, and performing a similarity calculation between the normalized sample category vector and the sentence vector, a text similarity calculation model is determined, the reliability of the text similarity calculation model is improved, and the technical problem of low reliability of the current model for calculating text similarity is solved.

[0069] Reference Figure 3 , Figure 3 It is a flowchart of the second embodiment of the method for generating a text similarity calculation model of the present invention.

[0070] Based on the above Figure 2 In the embodiment shown, in this embodiment, the step S10 specifically includes:

[0071] Step S11, setting the first sample category vector based on the number of categories included in the sample data set and the number of samples included in each category;

[0072] Step S12, normalizing the first sample category vector by L2 norm using a first preset formula to generate the second sample category vector, wherein the first preset formula is as follows:

[0073]

[0074] represents the i-th normalized category vector, C i represents the unnormalized category vector, is the length of the category vector.

[0075] Normalization is a way to simplify calculations, which is to transform a dimensional expression into a dimensionless expression, thus becoming a scalar.

[0076] For example, complex impedance can be normalized and written as: Z=R+jωL=R(1+jωL / R), and the complex part becomes a pure quantity without dimension.

[0077] In addition, many calculations in microwaves, such as circuit analysis, signal systems, and electromagnetic wave transmission, can be processed in this way, which not only ensures the convenience of calculations, but also highlights the essential meaning of physical quantities.

[0078] In statistics, the specific role of normalization is to summarize the statistical distribution of a sample. Normalization between 0 and 1 is the statistical probability distribution, and normalization between -1 and +1 is the statistical coordinate distribution.

[0079] Normalization definition: Normalization is to limit the data to be processed (through a certain algorithm) within a certain range you need. First of all, normalization is for the convenience of subsequent data processing, and secondly, it ensures faster convergence when the program is running. The specific role of normalization is to summarize the statistical distribution of unified samples. Normalization between 0-1 is a statistical probability distribution, and normalization in a certain interval is a statistical coordinate distribution. Normalization means sameness, unification and unity.

[0080] If it is a value on an interval, it can be normalized by using the relative position on the interval, that is, a phase reference point is selected, and the ratio of the relative position to the entire interval or the given value of the entire interval is used as the ratio to obtain a normalized data, such as a probability value 0<=p<=1;

[0081] If it is a physical quantity, it can generally be normalized after standardizing the weights and measures. If there is really no unified method, a custom concept can be given to describe it.

[0082] If it is a numerical value, many common mathematical functions can be used for normalization to make the comparability between them more obvious and stronger, such as logarithmic normalization, exponential normalization, trigonometric or inverse trigonometric function normalization, etc. The purpose of normalization may be to make incomparable data comparable, but still maintain the relative relationship between the two compared data, such as the size relationship, the big is still big, the small is still small, or for drawing, it is difficult to draw on a graph, and after normalization, it is very convenient to give the relative position on the graph, etc.;

[0083] From the perspective of sets, we can unify the dimensions, that is, abstract and normalize them, remove the attributes of unimportant and incomparable elements in the set, and retain those attributes that people care about. In this way, objects or things that were originally incomparable can be normalized, that is, classified into one category, and then they can be compared. In addition, people often like to use relative quantities for comparison. For example, the height and weight of humans and cows are not comparable, but the height / weight value may be comparable. How much a person eats and how much a cow eats may not be directly comparable, but relative to body weight, or relative to the amount of food needed to provide each person's energy in a day, they are comparable. From a mathematical point of view, these can be considered as turning dimensional quantities into dimensionless quantities.

[0084] This embodiment provides a method for generating a text similarity calculation model, the method obtaining a first sample category vector corresponding to a sample data set, and performing a normalization operation on the first sample category vector to generate a second sample category vector; obtaining a first sentence vector corresponding to a target sentence, and normalizing the sentence vector through a pre-trained language representation BERT model to generate a second sentence vector; calculating the similarity between the target sentence and the sample data set based on the second sentence vector and the second sample category vector; and generating a text similarity calculation model corresponding to the target sentence based on the similarity. In the above manner, by vectorizing the sample data and the target sentence, and performing a similarity calculation between the normalized sample category vector and the sentence vector, a text similarity calculation model is determined, the reliability of the text similarity calculation model is improved, and the technical problem of low reliability of the current model for calculating text similarity is solved.

[0085] Reference Figure 4 , Figure 4 It is a flowchart of the third embodiment of the method for generating a text similarity calculation model of the present invention.

[0086] Based on the above Figure 2 In the embodiment shown, in this embodiment, the step S20 specifically includes:

[0087] Step S21, inputting the target sentence into the BERT model, and using the output result as a Common Language Standard (CLS) tag;

[0088] Step S22, mapping the CLS label to the second sample category vector, and obtaining the first sentence vector based on the mapping result;

[0089] Step S23: normalize the first sentence vector by the L2 norm through a second preset formula to generate the second sentence vector.

[0090] In this embodiment, the first sentence vector is normalized by a second preset formula to generate the second sentence vector, wherein the second preset formula is as follows:

[0091]

[0092] represents the normalized vector of the second sentence, X represents the unnormalized vector of the first sentence, Represents the length of the sentence vector.

[0093] In a specific embodiment, the input of BERT can contain a sentence pair (sentence A and sentence B) or a single sentence. In addition, some flags with special functions are added:

[0094] The [CLS] sign is placed at the beginning of the first sentence, and the representation vector C obtained by BERT can be used for subsequent classification tasks.

[0095] This embodiment provides a method for generating a text similarity calculation model, the method obtaining a first sample category vector corresponding to a sample data set, and performing a normalization operation on the first sample category vector to generate a second sample category vector; obtaining a first sentence vector corresponding to a target sentence, and normalizing the sentence vector through a pre-trained language representation BERT model to generate a second sentence vector; calculating the similarity between the target sentence and the sample data set based on the second sentence vector and the second sample category vector; and generating a text similarity calculation model corresponding to the target sentence based on the similarity. In the above manner, by vectorizing the sample data and the target sentence, and performing a similarity calculation between the normalized sample category vector and the sentence vector, a text similarity calculation model is determined, the reliability of the text similarity calculation model is improved, and the technical problem of low reliability of the current model for calculating text similarity is solved.

[0096] Based on the above Figure 2 In the embodiment shown, in this embodiment, the step S30 specifically includes:

[0097] Based on the second sentence vector and the second sample category vector, the similarity is calculated by a third preset formula, wherein the third preset formula is as follows:

[0098]

[0099] The result calculated by the third preset formula is the similarity between the target sentence and the sample data set.

[0100] Based on all the above embodiments, this embodiment also includes:

[0101] The sample data set is generated, wherein the sample data set includes at least one category, and one category includes at least one sample.

[0102] Specifically, there is a data set D, which contains n categories and m samples in each category. Then we set the category vector C = {C1, C2, ..., Cn}, which consists of n vectors, and each vector has l dimensions.

[0103] This embodiment provides a method for generating a text similarity calculation model, the method obtaining a first sample category vector corresponding to a sample data set, and performing a normalization operation on the first sample category vector to generate a second sample category vector; obtaining a first sentence vector corresponding to a target sentence, and normalizing the sentence vector through a pre-trained language representation BERT model to generate a second sentence vector; calculating the similarity between the target sentence and the sample data set based on the second sentence vector and the second sample category vector; and generating a text similarity calculation model corresponding to the target sentence based on the similarity. In the above manner, by vectorizing the sample data and the target sentence, and performing a similarity calculation between the normalized sample category vector and the sentence vector, a text similarity calculation model is determined, the reliability of the text similarity calculation model is improved, and the technical problem of low reliability of the current model for calculating text similarity is solved.

[0104] Based on the above Figure 2 In the embodiment shown, in this embodiment, before step S40, the following steps are further included:

[0105] Through the similarity, a similarity matrix is ​​obtained, wherein the similarity matrix is ​​as follows:

[0106]

[0107] S is the similarity matrix, is the sentence vector, is the sample category vector, and S contains n vectors.

[0108] Based on the above Figure 2 In the embodiment shown, in this embodiment, after step S40, the following steps are further included:

[0109] The similarity matrix is ​​encoded with one-hot encoding to obtain the encoded true value, wherein the true value is as follows:

[0110] target = [0,0,1,0,...,0]

[0111] target is a real value array, the number of values ​​in the real value array is the same as the number of vectors in the similarity matrix, only one of them is 1 and the rest are 0.

[0112] Based on the above Figure 2 In the embodiment shown, in this embodiment, the step S40 specifically includes:

[0113] The loss function between the similarity matrix and the true value array is calculated by a preset loss function formula, and the text similarity calculation model is generated based on the minimum value of the loss function, wherein the preset loss function formula is as follows:

[0114]

[0115] L is the minimization value, and n is the number of vectors in the similarity matrix.

[0116] The present embodiment provides a method for generating a text similarity calculation model, the method obtaining a first sample category vector corresponding to a sample data set, and performing a normalization operation on the first sample category vector to generate a second sample category vector; obtaining a first sentence vector corresponding to a target sentence, and normalizing the sentence vector through a pre-trained language representation BERT model to generate a second sentence vector; calculating the similarity between the target sentence and the sample data set based on the second sentence vector and the second sample category vector; and generating a text similarity calculation model corresponding to the target sentence based on the similarity. In the above manner, by vectorizing the sample data and the target sentence, and performing a similarity calculation between the normalized sample category vector and the sentence vector, a text similarity calculation model is determined, the reliability of the text similarity calculation model is improved, and the technical problem of low reliability of the current model for calculating text similarity is solved.

[0117] In addition, an embodiment of the present invention also provides a device for generating a text similarity calculation model.

[0118] Reference Figure 5 , Figure 5 This is a functional module diagram of the first embodiment of the text similarity calculation model generation device of the present invention.

[0119] In this embodiment, the text similarity calculation model generation device includes:

[0120] The sample category vector generation module 10 is used to obtain a first sample category vector corresponding to the sample data set, and perform a normalization operation on the first sample category vector to generate a second sample category vector;

[0121] A sentence vector generation module 20 is used to obtain a first sentence vector corresponding to the target sentence, and normalize the sentence vector through a pre-trained language representation BERT model to generate a second sentence vector;

[0122] A similarity calculation module 30, configured to calculate the similarity between the target sentence and the sample data set according to the second sentence vector and the second sample category vector;

[0123] The model generation module 40 is used to generate a text similarity calculation model corresponding to the target sentence based on the similarity.

[0124] Furthermore, the sample category vector generation module 10 includes:

[0125] A first sample category vector unit, configured to set the first sample category vector based on the number of categories included in the sample data set and the number of samples included in each category;

[0126] The second sample category vector unit normalizes the first sample category vector by L2 norm through a first preset formula to generate the second sample category vector.

[0127] Furthermore, the sentence vector generation module 20 includes:

[0128] A CLS tag generation unit, used for inputting the target sentence into the BERT model, and using the output result as a common language specification CLS tag;

[0129] A CLS tag generation unit is used to input the target sentence into the BERT model and use the output result as a common language specification CLS tag

[0130] A first sentence vector unit, used to map the CLS label to the second sample category vector, and obtain the first sentence vector based on the mapping result;

[0131] The second sentence vector unit is used to normalize the first sentence vector by using the L2 norm to generate the second sentence vector.

[0132] Furthermore, the similarity calculation module 30 includes:

[0133] A similarity calculation unit is used to calculate the similarity based on the second sentence vector and the second sample category vector by a third preset formula, wherein the third preset formula is as follows:

[0134]

[0135] The result calculated by the third preset formula is the similarity between the target sentence and the sample data set.

[0136] Furthermore, the model generation module 40 includes:

[0137] The similarity matrix acquisition module is used to obtain a similarity matrix through the similarity, wherein the similarity matrix is ​​as follows:

[0138]

[0139] S is the similarity matrix, is the sentence vector, is the sample category vector, S contains n vectors;

[0140] The loss function between the similarity matrix and the true value array is calculated by a preset loss function formula, and the text similarity calculation model is generated based on the minimum value of the loss function, wherein the preset loss function formula is as follows:

[0141]

[0142] L is the minimization value, and n is the number of vectors in the similarity matrix.

[0143] Among them, each module in the above-mentioned text similarity calculation model generation device corresponds to each step in the above-mentioned text similarity calculation model generation method embodiment, and its function and implementation process will not be repeated here one by one.

[0144] In addition, an embodiment of the present invention further provides a computer-readable storage medium.

[0145] The computer-readable storage medium of the present invention stores a text similarity calculation model generation program, wherein when the text similarity calculation model generation program is executed by a processor, the steps of the text similarity calculation model generation method as described above are implemented.

[0146] The method implemented when the text similarity calculation model generation program is executed can refer to the various embodiments of the text similarity calculation model generation method of the present invention, and will not be repeated here.

[0147] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0148] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0149] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0150] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0151] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for generating a text similarity calculation model, characterized in that: The text similarity calculation model generation method comprises: Based on the number of categories included in the sample data set and the number of samples included in each category, a first sample category vector is set, and the first sample category vector is normalized by an L2 norm to generate a second sample category vector; Input the target sentence into the pre-trained language representation BERT model, use the output result as the common language specification CLS label, map the CLS label to the second sample category vector, and obtain a first sentence vector based on the mapping result, wherein the first sentence vector has the same dimension as the second sample category vector, and normalize the first sentence vector by the L2 norm to generate a second sentence vector; Calculating the similarity between the target sentence and the sample data set according to the second sentence vector and the second sample category vector; Based on the similarity, a text similarity calculation model corresponding to the target sentence is generated.

2. The text similarity calculation model generation method according to claim 1, characterized in that: The step of normalizing the first sample category vector to generate a second sample category vector includes: The first sample category vector is normalized by L2 norm through a first preset formula to generate the second sample category vector, wherein the first preset formula is as follows: represents the i-th normalized category vector, C i represents the unnormalized category vector, is the length of the category vector, n is the number of vectors in the category vector, l is the total number of dimensions of each vector in the category vector, and j is the current vector dimension in the analogy vector.

3. The text similarity calculation model generation method as claimed in claim 2, characterized in that: The step of normalizing the first sentence vector by using the L2 norm to generate a second sentence vector includes: The first sentence vector is normalized by the L2 norm through a second preset formula to generate the second sentence vector, wherein the second preset formula is as follows: represents the normalized vector of the second sentence, X represents the unnormalized vector of the first sentence, represents the length of the sentence vector, l is the total number of dimensions of each vector in the category vector, and j is the current vector dimension in the analogy vector.

4. The text similarity calculation model generation method according to claim 3, characterized in that: The calculating the similarity between the target sentence and the sample data set according to the second sentence vector and the second sample category vector includes: Based on the second sentence vector and the second sample category vector, the similarity is calculated by a third preset formula, wherein the third preset formula is as follows: l is the total number of dimensions of each vector in the category vector, and j is the current vector dimension in the analogy vector; The result calculated by the third preset formula is the similarity between the target sentence and the sample data set.

5. The text similarity calculation model generation method according to claim 1, characterized in that: Before generating a text similarity calculation model corresponding to the target sentence based on the similarity, the method includes: Through the similarity, a similarity matrix is ​​obtained, wherein the similarity matrix is ​​as follows: S is the similarity matrix, is the sentence vector, is the sample category vector, and S contains n vectors.

6. The method for generating a text similarity calculation model according to claim 5, wherein: After obtaining the similarity matrix through the similarity, the method further comprises: The similarity matrix is ​​encoded with one-hot encoding to obtain a true value array of the encoded similarity matrix, wherein the true value array is as follows: target = [0,0,1,0,...,0] target is a real value array, the number of values ​​in the real value array is the same as the number of vectors in the similarity matrix, only one of them is 1 and the rest are 0.

7. The method for generating a text similarity calculation model as claimed in claim 6, characterized in that: The step of generating a text similarity calculation model corresponding to the target sentence based on the similarity includes: The loss function between the similarity matrix and the true value array is calculated by a preset loss function formula, and the text similarity calculation model is generated based on the minimum value of the loss function, wherein the preset loss function formula is as follows: L is the minimization value, and n is the number of vectors in the similarity matrix.

8. A text similarity calculation model generation device, characterized in that: The text similarity calculation model generation device includes a sample category vector generation module, a sentence vector generation module, a similarity calculation module and a model generation module, the sample category vector generation module includes a first sample category vector unit and a second sample category vector unit, and the sentence vector generation module includes a CLS tag generation unit, a first sentence vector unit and a second sentence vector unit; A first sample category vector unit, used to set a first sample category vector based on the number of categories included in the sample data set and the number of samples included in each category; A second sample category vector unit, configured to perform a normalization operation on the first sample category vector by using an L2 norm to generate a second sample category vector; A CLS label generation unit is used to input the target sentence into the pre-trained language representation BERT model and use the output result as the common language specification CLS label; A first sentence vector unit, used to map the CLS label to the second sample category vector, and obtain a first sentence vector based on the mapping result, wherein the first sentence vector has the same dimension as the second sample category vector; A second sentence vector unit, used to normalize the first sentence vector by the L2 norm to generate a second sentence vector; A similarity calculation module, used for calculating the similarity between the target sentence and the sample data set according to the second sentence vector and the second sample category vector; The model generation module is used to generate a text similarity calculation model corresponding to the target sentence based on the similarity.

9. A text similarity calculation model generation device, characterized in that: The text similarity calculation model generation device includes a processor, a memory, and a text similarity calculation model generation program stored in the memory and executable by the processor, wherein when the text similarity calculation model generation program is executed by the processor, the steps of the text similarity calculation model generation method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a text similarity calculation model generation program, wherein when the text similarity calculation model generation program is executed by a processor, the steps of the text similarity calculation model generation method according to any one of claims 1 to 7 are implemented.

Images