Model training method, sentence vector generation method, equipment and storage medium

Abstract

The invention provides a model training method, a sentence vector generation method, equipment and a storage medium, and belongs to the field of computers. Initializing a preset neural network model to obtain a sentence vector generation model, and selecting one piece of sample data from the sample data set as target sample data; inputting a plurality of text statements in the target sample data into a coding layer for text vectorization to obtain a first vector set; inputting the token vector corresponding to each text statement in the first vector set into a decoding layer for cyclic dilated convolution processing to obtain a predicted sentence vector corresponding to each text statement; and circularly executing the previous steps until the sentence vector generation model converges to obtain a target sentence vector generation model. According to the invention, the accurate and stable target vector generation model can be generated. The invention further relates to the technical field of block chains, and the target sentence vector generation model can be stored in the block chain.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, a sentence vector generation method, a device and a storage medium. Background Art

[0002] Voice quality inspection is a key link in ensuring the quality of customer service, especially in industries such as finance, insurance, and customer service. Existing technologies usually rely on manual monitoring and evaluation, or use rule-based automated systems for voice analysis. These methods are inefficient, costly, or inaccurate when processing large-scale data. Therefore, the concept of sentence embedding has been introduced, which maps the semantic information between words and characters in a sentence into a quantifiable space to obtain a sentence embedding. The generated sentence embedding is usually provided to downstream tasks for further processing, such as similarity calculation, classification, and clustering based on sentence embedding.

[0003] Some existing sentence vector generation technologies simply segment the words in a sentence, convert them into word vectors, and then calculate the average to obtain the sentence vector. However, this will lose the semantic information in the sentence and affect the accuracy of the sentence vector. Therefore, how to design a model that can accurately identify sentence vectors to improve the accuracy of sentence vector conversion and thus improve the efficiency and accuracy of semantic analysis is a problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of this application is to provide a model training method, a sentence vector generation method, a device and a storage medium, aiming to train a model with accurate sentence vector recognition to improve the accuracy of sentence vector conversion, thereby improving the efficiency and accuracy of semantic analysis.

[0005] In a first aspect, the present application provides a model training method, the model training method comprising the following steps:

[0006] Acquire a sample data set, the sample data set comprising a plurality of sample data, the sample data comprising a plurality of text sentences and annotated sentence vectors corresponding to each text sentence;

[0007] Initializing a preset neural network model to obtain a sentence vector generation model, and selecting a sample data from the sample data set as target sample data, wherein the sentence vector generation model includes an encoding layer and a decoding layer;

[0008] Inputting multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, wherein the first vector set includes at least one token vector corresponding to each text sentence;

[0009] Inputting the token vector corresponding to each of the text sentences in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each of the text sentences;

[0010] Determining whether the sentence vector generation model converges according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences;

[0011] If the sentence vector generation model has not converged, return to the loop to execute the step of selecting a sample data from the sample data set as the target sample data until the sentence vector generation model converges, and determine the latest sentence vector generation model as the target sentence vector generation model.

[0012] In a second aspect, the present application further provides a sentence vector generation method, the sentence vector generation method comprising:

[0013] Get the text to be recognized;

[0014] The text to be recognized is input into the sentence vector generation model for sentence vector extraction to generate a target sentence vector corresponding to the text to be recognized, wherein the sentence vector generation model is trained according to the model training method described in the embodiment of the present application.

[0015] In a third aspect, the present application further provides a model training device, the model training device comprising an acquisition module, a generation module and a determination module, wherein:

[0016] The acquisition module is used to acquire a sample data set, wherein the sample data set includes a plurality of sample data, and the sample data includes a plurality of text sentences and annotated sentence vectors corresponding to each text sentence;

[0017] The generation module is used to initialize the preset neural network model to obtain a sentence vector generation model, wherein the sentence vector generation model includes an encoding layer and a decoding layer;

[0018] The acquisition module is further used to select a sample data from the sample data set as the target sample data;

[0019] The generating module is further used to input multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, wherein the first vector set includes at least one token vector corresponding to each text sentence;

[0020] The generating module is further used to input the token vector corresponding to each of the text sentences in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each of the text sentences;

[0021] The determination module is used to determine whether the sentence vector generation model converges according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences;

[0022] The generation module is used to continue to execute the step of selecting a sample data from the sample data set as the target sample data if the sentence vector generation model has not converged until the sentence vector generation model converges.

[0023] In a fourth aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the model training method as described above and / or the steps of the sentence vector generation method as described above are implemented.

[0024] In a fifth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the model training method as described above and / or the steps of the sentence vector generation method as described above are implemented.

[0025] The present application provides a model training method, a sentence vector generation method, a device and a storage medium. The present application obtains a sample data set, wherein the sample data set includes multiple sample data, and the sample data includes multiple text sentences and annotated sentence vectors corresponding to each text sentence; initializes a preset neural network model to obtain a sentence vector generation model, and selects a sample data from the sample data set as target sample data, wherein the sentence vector generation model includes an encoding layer and a decoding layer; inputs multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, wherein the first vector set includes at least one token vector corresponding to each text sentence; inputs the token vector corresponding to each text sentence in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each text sentence; determines whether the sentence vector generation model converges according to the predicted sentence vector and the annotated sentence vector corresponding to each of the text sentences; returns to the loop to execute the step of selecting a sample data from the sample data set as the target sample data until the sentence vector generation model converges, and determines the latest sentence vector generation model as the target sentence vector generation model. In the present application, by obtaining a sample data set and performing cyclic void convolution training on each target sample data in the sample data set, the predicted sentence vector corresponding to each sentence can be accurately obtained. According to the predicted sentence vector and the labeled sentence vector corresponding to each sentence, it is possible to accurately determine whether the sentence vector generation model has converged. When the sentence vector generation model has not converged, the model parameters are updated and the target sample data training is continued until the model converges. An accurate and stable target vector generation model can be generated, which can effectively improve the efficiency and accuracy of the target vector generation model in converting text into sentence vectors, thereby improving the efficiency and accuracy of semantic analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A flowchart of a model training method provided in an embodiment of the present application;

[0028] Figure 2 for Figure 1 A flowchart of the sub-steps of the model training method in ;

[0029] Figure 3 A flowchart of a sentence vector generation method provided in an embodiment of the present application;

[0030] Figure 4A schematic block diagram of a model training device provided in an embodiment of the present application;

[0031] Figure 5 for Figure 4 A schematic block diagram of submodules of a model training device in;

[0032] Figure 6 A schematic block diagram of a sentence vector generation device provided in an embodiment of the present application;

[0033] Figure 7 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0034] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0037] Voice quality inspection is a key link in ensuring the quality of customer service, especially in industries such as finance, insurance, and customer service. Existing technologies usually rely on manual monitoring and evaluation, or use rule-based automated systems for voice analysis. These methods are inefficient, costly, or inaccurate when processing large-scale data. Therefore, the concept of sentence embedding has been introduced, which maps the semantic information between words and characters in a sentence into a quantifiable space to obtain a sentence embedding. The generated sentence embedding is usually provided to downstream tasks for further processing, such as similarity calculation, classification, and clustering based on sentence embedding.

[0038] Some existing sentence vector generation technologies simply segment the words in a sentence separately, convert the words into word vectors, and then calculate the average value to obtain the sentence vector. However, this will lose the semantic information in the sentence and affect the accuracy of the sentence vector.

[0039] To solve the aforementioned problems, the present application provides a model training method, a sentence vector generation method, a device and a storage medium. The model training method includes: obtaining a sample data set, the sample data set includes multiple sample data, the sample data includes multiple text sentences and annotated sentence vectors corresponding to each text sentence; initializing a preset neural network model to obtain a sentence vector generation model, and selecting a sample data from the sample data set as target sample data, the sentence vector generation model includes an encoding layer and a decoding layer; inputting multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, the first vector set includes at least one token vector corresponding to each text sentence; inputting the token vector corresponding to each of the text sentences in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each of the text sentences; determining whether the sentence vector generation model converges according to the predicted sentence vector and the annotated sentence vector corresponding to each of the text sentences; if the sentence vector generation model does not converge, returning to the loop to execute the step of selecting a sample data from the sample data set as the target sample data until the sentence vector generation model converges, and determining the latest sentence vector generation model as the target sentence vector generation model.

[0040] Among them, the model training method can be applied to computer devices, which can be electronic devices such as mobile phones, tablet computers, laptops, desktop computers, personal digital assistants and wearable devices.

[0041] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0042] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0043] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0044] Please refer to Figure 1 , Figure 1A flowchart of a model training method provided in an embodiment of the present application.

[0045] like Figure 1 As shown, the model training method includes steps S101 to S106.

[0046] Step S101: Obtain a sample data set, where the sample data set includes a plurality of sample data, and the sample data includes a plurality of text sentences and annotated sentence vectors corresponding to each text sentence.

[0047] The sample data set includes multiple sample data, each sample data includes multiple text sentences and annotated sentence vectors corresponding to each text sentence.

[0048] In some embodiments, a speech data set is obtained, and a speech data is selected from the speech data set as target speech data, and the speech data set includes multiple speech data; the target speech data is input into a preset speech recognition model to obtain a target text corresponding to the target speech data, and the speech recognition model is a pre-trained neural network model; the target speech data is input into a preset sentence vector recognition model to obtain a sentence vector set corresponding to the target speech data, and the sentence vector set includes a sentence vector corresponding to each speech, and the preset sentence vector recognition model is a neural network model pre-trained based on speech data; the target text and the sentence vector set are associated to generate sample data, and the step of selecting a speech data from the speech data set as the target speech data is continued to generate a sample data set. By obtaining a speech data set and performing sentence vector conversion on each speech data in the speech data set, a sample data set can be accurately obtained.

[0049] It should be noted that the neural network model can be selected according to actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the neural network model can be a convolutional neural network model and a recurrent convolutional neural network model. The speech data training to generate a preset speech recognition model is a conventional model training process, and the embodiment of the present invention does not elaborate on this.

[0050] It should be noted that the preset sentence vector recognition model is a neural network model pre-trained based on speech data, specifically: obtain a sample speech data set, the sample speech data set includes multiple sample speech data, the sample speech data includes at least one speech and a sentence vector corresponding to each speech; select a sample speech data from the sample speech data set as the target sample speech data; input the target sample speech data into the preset neural network model to obtain the predicted sentence vector corresponding to each speech in the target sample speech data; determine whether the neural network model converges according to the predicted sentence vector and the labeled sentence vector corresponding to each speech in the target sample speech data, if the neural network model does not converge, then continue to execute the step of selecting a sample speech data from the sample speech data set as the target sample speech data until the neural network model converges to obtain the preset sentence vector recognition model. Training the neural network model based on the sample speech data set can accurately obtain the preset sentence vector recognition model.

[0051] It should be noted that the training process of the preset sentence vector recognition model can refer to the training process of the sentence vector generation model in the embodiment of the present application, and the present application will not go into details about this.

[0052] In some embodiments, historical text sentence vector data is obtained, and the historical text sentence vector data is a plurality of text sentences generated before the current timestamp and the sentence vectors corresponding to each text sentence. A sample data set is constructed based on the sentence vectors corresponding to the plurality of text sentences and each text sentence in the historical text sentence vector data. The sample data set can be quickly constructed through the historical text sentence vector data, which greatly improves the efficiency of model training.

[0053] Step S102: Initialize the preset neural network model to obtain a sentence vector generation model, and select a sample data from the sample data set as the target sample data, wherein the sentence vector generation model includes a coding layer and a decoding layer.

[0054] Among them, the preset neural network model includes but is not limited to a convolutional neural network model and a recurrent convolutional neural network model, etc.

[0055] In some embodiments, a preset neural network model is obtained and initialized to obtain a sentence vector generation model, wherein the sentence vector generation model includes an encoding layer and a decoding layer.

[0056] In some embodiments, a sample data is selected from the sample data set as the target sample data. The method of selecting the sample data can be set according to actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the method of selecting the sample data can be a random selection method.

[0057] Step S103: input multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, wherein the first vector set includes at least one token vector corresponding to each text sentence.

[0058] Among them, the first vector set includes at least one token vector for vectorizing each text sentence. The token is the smallest discrete unit. The token can be set according to actual conditions, and the embodiment of the present invention does not make specific limitations on this. For example, the token can be a word.

[0059] In some embodiments, multiple text sentences in the target sample data are input into the encoding layer for text vectorization to obtain at least one token vector corresponding to each text speech. The first vector set can be accurately obtained by vectorizing each text sentence through the encoding layer.

[0060] Step S104: input the token vector corresponding to each of the text sentences in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each of the text sentences.

[0061] The predicted sentence vector is a vector obtained by predicting each text sentence through a sentence vector generation model.

[0062] In some embodiments, Figure 2 As shown, step S104 includes sub-steps S1041 and S1042.

[0063] Sub-step S1041: grouping multiple token vectors in each of the text sentences according to a preset number of token vectors to obtain multiple token vector groups corresponding to each of the text sentences.

[0064] The number of preset token vectors may be set according to actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the number of preset token vectors may be set to 2.

[0065] Exemplarily, the target sample data includes a first text sentence, a second text sentence, a third text sentence, and a fourth text sentence; the first text sentence, the second text sentence, the third text sentence, and the fourth text sentence are vectorized to obtain a first text sentence including token vector 1, token vector 2, and token vector 3, a second text sentence including token vector 4, token vector 5, token vector 6, token vector 7, and token vector 8, a third text sentence including token vector 9, token vector 10, and token vector 11, and a fourth text sentence including token vector 12, token vector 13, token vector 14, and token vector 15; a plurality of token vectors in each text sentence are grouped according to a preset number of token vectors, wherein the preset number of token vectors is 2, to obtain token vector group 1 and token vector group 2 corresponding to the first text sentence, and token vector group 3 corresponding to the second text sentence. The token vector groups 3, 4 and 5 corresponding to the statement, the token vector groups 6 and 7 corresponding to the third text statement, and the token vector groups 8 and 9 corresponding to the fourth text statement; wherein, token vector group 1 includes token vector 1 and token vector 2, token vector group 2 includes token vector 3, token vector group 3 includes token vector 4 and token vector 5, token vector group 4 includes token vector 6 and token vector 7, token vector group 5 includes token vector 8, token vector group 6 includes token vector 9 and token vector 10, token vector group 7 includes token vector 11, token vector group 8 includes token vector 12 and token vector 13, and token vector group 9 includes token vector 14 and token vector 15.

[0066] Sub-step S1042: performing circular atrous convolution processing on multiple token vector groups corresponding to each of the text sentences to obtain a predicted sentence vector corresponding to each of the text sentences.

[0067] In some embodiments, each of the token vector groups of each text sentence is subjected to a dilated convolution process to obtain a target vector corresponding to each token vector group; at least one target vector in each text sentence is grouped with a preset number of vectors to obtain multiple target vector groups corresponding to each text sentence; each target vector group of each text sentence is subjected to a dilated convolution process to obtain an updated target vector corresponding to each target vector group; it is determined whether the dilated convolution of the text sentence meets the preset requirements. If the dilated convolution of the text sentence does not meet the preset requirements, the step of grouping at least one of the target vectors in each text sentence with a preset number of vectors to obtain multiple target vector groups corresponding to each text sentence is returned to the step of grouping at least one of the target vectors in each text sentence with a preset number of vectors to obtain multiple target vector groups corresponding to each text sentence until the dilated convolution of the text sentence meets the preset requirements to obtain the predicted sentence vector corresponding to each text sentence. The preset requirements can be set according to the actual situation. For example, the preset requirements can be the number of dilated convolutions, which can be 5 times. The preset number of vectors can be set according to the actual situation, and the embodiment of the present invention does not specifically limit this. For example, the preset number of vectors can be 2.

[0068] Step S105: Determine whether the sentence vector generation model converges according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences.

[0069] In some embodiments, the loss value of the sentence vector generation model is determined based on the predicted sentence vector and the annotated sentence vector corresponding to each text sentence; if the loss value is less than or equal to the preset loss value, it is determined that the sentence vector generation model has converged; if the loss value is greater than the preset loss value, it is determined that the sentence vector generation model has not converged. Among them, the preset loss value can be set according to the actual situation, and the embodiment of the present invention does not make specific limitations on this. For example, the preset loss value can be set to 0.02. According to the loss value of the model, it is possible to accurately know whether the model has converged, greatly improving the efficiency and accuracy of the training model.

[0070] In some embodiments, the loss value of the sentence vector generation model can be determined based on the predicted sentence vector and the labeled sentence vector corresponding to each text sentence: performing similarity calculations on the predicted sentence vector and the labeled sentence vector of the same text sentence in sequence to obtain multiple similarity values; performing a mean operation on each similarity value to obtain a target similarity value, and subtracting the target similarity value from the unit 1 to obtain the loss value of the sentence vector generation model.

[0071] Step S106: If the sentence vector generation model has not converged, return to the loop to execute the step of selecting a sample data from the sample data set as the target sample data until the sentence vector generation model converges, and determine the latest sentence vector generation model as the target sentence vector generation model.

[0072] If the sentence vector generation model has not converged, return to the loop execution to select a sample data from the sample data set as the target sample data; input multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, which includes at least one token vector corresponding to each text sentence; input the token vector corresponding to each text sentence in the first vector set into the decoding layer for cyclic hole convolution processing to obtain the predicted sentence vector corresponding to each text sentence; determine whether the sentence vector generation model has converged based on the predicted sentence vector and the labeled sentence vector corresponding to each text sentence; until the sentence vector generation model converges, the latest sentence vector generation model is determined as the target sentence vector generation model. By sequentially training the sample data, a converged target sentence vector generation model can be obtained.

[0073] The model training method provided in the above embodiment obtains a sample data set, the sample data set includes multiple sample data, the sample data includes multiple text sentences and annotated sentence vectors corresponding to each text sentence; initializes a preset neural network model to obtain a sentence vector generation model, and selects a sample data from the sample data set as the target sample data, the sentence vector generation model includes an encoding layer and a decoding layer; inputs multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, the first vector set includes at least one token vector corresponding to each text sentence; inputs the token vector corresponding to each text sentence in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each text sentence; determines whether the sentence vector generation model converges according to the predicted sentence vector and the annotated sentence vector corresponding to each of the text sentences; returns to the loop to execute the step of selecting a sample data from the sample data set as the target sample data until the sentence vector generation model converges, and determines the latest sentence vector generation model as the target sentence vector generation model. In the present application, by obtaining a sample data set and performing cyclic void convolution training on each target sample data in the sample data set, the predicted sentence vector corresponding to each sentence can be accurately obtained. According to the predicted sentence vector and the labeled sentence vector corresponding to each sentence, it is possible to accurately determine whether the sentence vector generation model has converged. When the sentence vector generation model has not converged, the model parameters are updated and the target sample data training is continued until the model converges. An accurate and stable target vector generation model can be generated, which can effectively improve the efficiency and accuracy of the target vector generation model in converting text into sentence vectors, thereby improving the efficiency and accuracy of semantic analysis.

[0074] See also Figure 3 , Figure 3 A flowchart of a sentence vector generation method provided in an embodiment of the present application.

[0075] like Figure 3As shown, the sentence vector generation method includes steps S201 to S202.

[0076] Step S201: Obtain text to be recognized.

[0077] Get the text to be recognized, which can be a sentence or a paragraph.

[0078] Step S202: input the text to be recognized into the target sentence vector generation model to extract sentence vectors, and generate a target sentence vector corresponding to the text to be recognized.

[0079] By extracting the vector of each text sentence in the text to be recognized through the target sentence vector generation model, the target sentence vector corresponding to each sentence in the text to be recognized can be accurately obtained, thereby improving the efficiency and accuracy of semantic analysis.

[0080] The above sentence vector generation method, by acquiring the text to be recognized and inputting the text to be recognized into the sentence vector generation model for sentence vector extraction, can accurately obtain the target sentence vector corresponding to each sentence in the text to be recognized, greatly improving the accuracy and efficiency of sentence vectorization, thereby improving the efficiency and accuracy of semantic analysis.

[0081] See also Figure 4 , Figure 4 A schematic block diagram of a model training device provided in an embodiment of the present application.

[0082] like Figure 4 As shown, the model training device 300 includes an acquisition module 310, a generation module 320 and a determination module 330, wherein:

[0083] The acquisition module 310 is used to acquire a sample data set, wherein the sample data set includes a plurality of sample data, and the sample data includes a plurality of text sentences and annotated sentence vectors corresponding to each text sentence;

[0084] The generation module 320 is used to initialize the preset neural network model to obtain a sentence vector generation model, wherein the sentence vector generation model includes an encoding layer and a decoding layer;

[0085] The acquisition module 310 is further used to select a sample data from the sample data set as target sample data;

[0086] The generating module 320 is further configured to input multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, wherein the first vector set includes at least one token vector corresponding to each text sentence;

[0087] The generating module 320 is further configured to input the token vector corresponding to each of the text sentences in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each of the text sentences;

[0088] The determination module 330 is used to determine whether the sentence vector generation model converges according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences;

[0089] The generation module 320 is used to return to the step of looping and selecting a sample data from the sample data set as the target sample data if the sentence vector generation model has not converged, until the sentence vector generation model converges, and determine the latest sentence vector generation model as the target sentence vector generation model.

[0090] In some embodiments, Figure 5 As shown, the generating module 320 includes a first generating submodule 321 and a second generating submodule 322, wherein:

[0091] The first generating submodule 321 is used to group multiple token vectors in each of the text sentences according to a preset number of token vectors to obtain multiple token vector groups corresponding to each of the text sentences;

[0092] The second generating submodule 322 is used to perform cyclic hole convolution processing on multiple token vector groups corresponding to each of the text sentences to obtain a predicted sentence vector corresponding to each of the text sentences.

[0093] In some embodiments, the second generating submodule 322 is further used to:

[0094] Performing a dilated convolution process on each of the token vector groups of each of the text sentences to obtain a target vector corresponding to each of the token vector groups;

[0095] Grouping at least one of the target vectors in each of the text sentences according to a preset number of vectors to obtain a plurality of target vector groups corresponding to each of the text sentences;

[0096] Performing a dilated convolution process on each target vector group of each text sentence to obtain an updated target vector corresponding to each target vector group;

[0097] Determine whether the dilated convolution of the text sentence meets the preset requirements. If the dilated convolution of the text sentence does not meet the preset requirements, return to the step of grouping at least one of the target vectors in each of the text sentences by a preset number of token vectors to obtain a plurality of target vector groups corresponding to each of the text sentences, until the dilated convolution of the text sentence meets the preset requirements, and obtains a predicted sentence vector corresponding to each of the text sentences.

[0098] In some embodiments, the determining module 330 is further configured to:

[0099] Determining a loss value of the sentence vector generation model according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences;

[0100] If the loss value is less than or equal to a preset loss value, it is determined that the sentence vector generation model has converged;

[0101] If the loss value is greater than a preset loss value, it is determined that the sentence vector generation model has not converged.

[0102] Sequentially calculating similarities between the predicted sentence vector and the annotated sentence vector of the same text sentence to obtain multiple similarity values;

[0103] A mean operation is performed on each of the similarity values ​​to obtain a target similarity value, and the target similarity value is subtracted from the unit 1 to obtain a loss value of the sentence vector generation model.

[0104] In some embodiments, the acquisition module 310 is further used to:

[0105] Acquire a speech data set, and select a speech data from the speech data set as target speech data, wherein the speech data set comprises a plurality of speech data;

[0106] Inputting the target speech data into a preset speech recognition model to obtain a target text corresponding to the target speech data, wherein the preset speech recognition model is a pre-trained neural network model;

[0107] Input the target speech data into a preset sentence vector recognition model to obtain a sentence vector set corresponding to the target speech data, wherein the sentence vector set includes a sentence vector corresponding to each speech, and the preset sentence vector recognition model is a neural network model pre-trained based on speech data;

[0108] The target text and the sentence vector set are associated to generate sample data, and the step of selecting a speech data as the target speech data from the speech data set is continued to generate a sample data set.

[0109] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-mentioned model training device can refer to the corresponding process in the aforementioned model training method embodiment, and will not be repeated here.

[0110] See also Figure 6 , Figure 6 A schematic block diagram of a sentence vector generation device provided in an embodiment of the present application.

[0111] like Figure 6 As shown, the sentence vector generating device 400 includes an acquisition module 410 and a generation module 420, wherein:

[0112] The acquisition module 410 is used to acquire the text to be recognized;

[0113] The generation module 420 is used to input the text to be recognized into the target sentence vector generation model to extract the sentence vector and generate the target sentence vector corresponding to the text to be recognized.

[0114] It should be noted that, those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-mentioned sentence vector generation device can refer to the corresponding process in the aforementioned sentence vector generation method embodiment, and will not be repeated here.

[0115] See also Figure 7 , Figure 7 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0116] like Figure 7 As shown, the computer device 500 includes a processor 502 and a memory 503 connected via a system bus 501 , wherein the memory 503 may include a storage medium and an internal memory.

[0117] The storage medium may store a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor may execute any one of the model training methods and / or sentence vector generation methods.

[0118] The processor 502 is used to provide computing and control capabilities and support the operation of the entire computer device.

[0119] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any model training method.

[0120] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] It should be understood that the processor 502 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0122] In one embodiment, the processor 502 is used to run a computer program stored in the memory to implement the following steps:

[0123] Acquire a sample data set, the sample data set comprising a plurality of sample data, the sample data comprising a plurality of text sentences and annotated sentence vectors corresponding to each text sentence;

[0124] Initializing a preset neural network model to obtain a sentence vector generation model, and selecting a sample data from the sample data set as target sample data, wherein the sentence vector generation model includes an encoding layer and a decoding layer;

[0125] Inputting multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, wherein the first vector set includes at least one token vector corresponding to each text sentence;

[0126] Inputting the token vector corresponding to each of the text sentences in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each of the text sentences;

[0127] Determining whether the sentence vector generation model converges according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences;

[0128] If the sentence vector generation model has not converged, return to the loop to execute the step of selecting a sample data from the sample data set as the target sample data until the sentence vector generation model converges, and determine the latest sentence vector generation model as the target sentence vector generation model.

[0129] In one embodiment, when the processor 502 implements the inputting of the token vector corresponding to each of the text sentences in the first vector set into the decoding layer for cyclic hole convolution processing to obtain the predicted sentence vector corresponding to each of the text sentences, it is used to implement:

[0130] Grouping multiple token vectors in each of the text sentences according to a preset number of token vectors to obtain multiple token vector groups corresponding to each of the text sentences;

[0131] A circular atrous convolution process is performed on multiple token vector groups corresponding to each of the text sentences to obtain a predicted sentence vector corresponding to each of the text sentences.

[0132] In one embodiment, the processor 502, when performing the cyclic hole convolution processing on the multiple token vector groups corresponding to each of the text sentences to obtain the predicted sentence vector corresponding to each of the text sentences, is used to implement:

[0133] Performing a dilated convolution process on each of the token vector groups of each of the text sentences to obtain a target vector corresponding to each of the token vector groups;

[0134] Grouping at least one of the target vectors in each of the text sentences according to a preset number of vectors to obtain a plurality of target vector groups corresponding to each of the text sentences;

[0135] Performing a dilated convolution process on each target vector group of each text sentence to obtain an updated target vector corresponding to each target vector group;

[0136] Determine whether the dilated convolution of the text sentence meets the preset requirements. If the dilated convolution of the text sentence does not meet the preset requirements, return to the step of grouping at least one of the target vectors in each of the text sentences by a preset number of token vectors to obtain a plurality of target vector groups corresponding to each of the text sentences, until the dilated convolution of the text sentence meets the preset requirements, and obtains a predicted sentence vector corresponding to each of the text sentences.

[0137] In one embodiment, when the processor 502 determines whether the sentence vector generation model converges according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences, it is configured to implement:

[0138] Determining a loss value of the sentence vector generation model according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences;

[0139] If the loss value is less than or equal to a preset loss value, it is determined that the sentence vector generation model has converged;

[0140] If the loss value is greater than a preset loss value, it is determined that the sentence vector generation model has not converged.

[0141] In one embodiment, when the processor 502 determines the loss value of the sentence vector generation model according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences, it is configured to implement:

[0142] Sequentially calculating similarities between the predicted sentence vector and the annotated sentence vector of the same text sentence to obtain multiple similarity values;

[0143] A mean operation is performed on each of the similarity values ​​to obtain a target similarity value, and the target similarity value is subtracted from the unit 1 to obtain a loss value of the sentence vector generation model.

[0144] In one embodiment, when implementing the acquiring of the sample data set, the processor 502 is configured to implement:

[0145] Acquire a speech data set, and select a speech data from the speech data set as target speech data, wherein the speech data set comprises a plurality of speech data;

[0146] Inputting the target speech data into a preset speech recognition model to obtain a target text corresponding to the target speech data, wherein the preset speech recognition model is a pre-trained neural network model;

[0147] Input the target speech data into a preset sentence vector recognition model to obtain a sentence vector set corresponding to the target speech data, wherein the sentence vector set includes a sentence vector corresponding to each speech, and the preset sentence vector recognition model is a neural network model pre-trained based on speech data;

[0148] The target text and the sentence vector set are associated to generate sample data, and the step of selecting a speech data as the target speech data from the speech data set is continued to generate a sample data set.

[0149] In one embodiment, the processor 502 is further configured to implement:

[0150] Get the text to be recognized;

[0151] The text to be recognized is input into the target sentence vector generation model to extract the sentence vector, and generate the target sentence vector corresponding to the text to be recognized.

[0152] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the computer device described above can refer to the corresponding process in the aforementioned model training method and / or sentence vector generation method embodiment, and will not be repeated here.

[0153] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the model training method and / or sentence vector generation method of the present application.

[0154] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may be non-volatile or volatile. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0155] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0156] The blockchain referred to in the present invention is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the blockchain underlying platform, platform product service layer, and application service layer.

[0157] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0158] It should also be understood that the term "and / or" used in the present specification refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0159] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application.

Claims

1. A model training method, characterized in that: include: Acquire a sample data set, the sample data set comprising a plurality of sample data, the sample data comprising a plurality of text sentences and annotated sentence vectors corresponding to each text sentence; Initializing a preset neural network model to obtain a sentence vector generation model, and selecting a sample data from the sample data set as target sample data, wherein the sentence vector generation model includes an encoding layer and a decoding layer; Inputting multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, wherein the first vector set includes at least one token vector corresponding to each text sentence; Inputting the token vector corresponding to each of the text sentences in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each of the text sentences; Determining whether the sentence vector generation model converges according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences; If the sentence vector generation model has not converged, return to the loop to execute the step of selecting a sample data from the sample data set as the target sample data until the sentence vector generation model converges, and determine the latest sentence vector generation model as the target sentence vector generation model.

2. The model training method according to claim 1, characterized in that: The step of inputting the token vector corresponding to each of the text sentences in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each of the text sentences includes: Grouping multiple token vectors in each of the text sentences according to a preset number of token vectors to obtain multiple token vector groups corresponding to each of the text sentences; A circular atrous convolution process is performed on multiple token vector groups corresponding to each of the text sentences to obtain a predicted sentence vector corresponding to each of the text sentences.

3. The model training method according to claim 2, characterized in that: The step of performing cyclic atrous convolution processing on the multiple token vector groups corresponding to each of the text sentences to obtain a predicted sentence vector corresponding to each of the text sentences includes: Performing a dilated convolution process on each of the token vector groups of each of the text sentences to obtain a target vector corresponding to each of the token vector groups; Grouping at least one of the target vectors in each of the text sentences according to a preset number of vectors to obtain a plurality of target vector groups corresponding to each of the text sentences; Performing a dilated convolution process on each target vector group of each text sentence to obtain an updated target vector corresponding to each target vector group; Determine whether the dilated convolution of the text sentence meets the preset requirements. If the dilated convolution of the text sentence does not meet the preset requirements, return to the step of grouping at least one of the target vectors in each of the text sentences by a preset number of vectors to obtain a plurality of target vector groups corresponding to each of the text sentences, until the dilated convolution of the text sentence meets the preset requirements, and obtains a predicted sentence vector corresponding to each of the text sentences.

4. The model training method according to claim 1, characterized in that: The step of determining whether the sentence vector generation model has converged according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences includes: Determining a loss value of the sentence vector generation model according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences; If the loss value is less than or equal to a preset loss value, it is determined that the sentence vector generation model has converged; If the loss value is greater than a preset loss value, it is determined that the sentence vector generation model has not converged.

5. The model training method according to claim 4, characterized in that: The step of determining the loss value of the sentence vector generation model according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences includes: Sequentially calculating similarities between the predicted sentence vector and the annotated sentence vector of the same text sentence to obtain multiple similarity values; A mean operation is performed on each of the similarity values ​​to obtain a target similarity value, and the target similarity value is subtracted from the unit 1 to obtain a loss value of the sentence vector generation model.

6. The model training method according to claim 1, characterized in that: The obtaining of the sample data set comprises: Acquire a speech data set, and select a speech data from the speech data set as target speech data, wherein the speech data set comprises a plurality of speech data; Inputting the target speech data into a preset speech recognition model to obtain a target text corresponding to the target speech data, wherein the preset speech recognition model is a pre-trained neural network model; Input the target speech data into a preset sentence vector recognition model to obtain a sentence vector set corresponding to the target speech data, wherein the sentence vector set includes a sentence vector corresponding to each speech, and the preset sentence vector recognition model is a neural network model pre-trained based on speech data; The target text and the sentence vector set are associated to generate sample data, and the step of selecting a speech data as the target speech data from the speech data set is continued to generate a sample data set.

7. A sentence vector generation method, characterized in that: include: Get the text to be recognized; The text to be recognized is input into a target sentence vector generation model for sentence vector extraction to generate a target sentence vector corresponding to the text to be recognized, wherein the target sentence vector generation model is trained by the model training method according to any one of claims 1 to 6.

8. A model training device, characterized in that: The model training device includes an acquisition module, a generation module and a determination module, wherein: The acquisition module is used to acquire a sample data set, wherein the sample data set includes a plurality of sample data, and the sample data includes a plurality of text sentences and annotated sentence vectors corresponding to each text sentence; The generation module is used to initialize the preset neural network model to obtain a sentence vector generation model, wherein the sentence vector generation model includes an encoding layer and a decoding layer; The acquisition module is further used to select a sample data from the sample data set as the target sample data; The generating module is further used to input multiple text sentences in the target sample data into the encoding layer for text vectorization to obtain a first vector set, wherein the first vector set includes at least one token vector corresponding to each text sentence; The generating module is further used to input the token vector corresponding to each of the text sentences in the first vector set into the decoding layer for cyclic hole convolution processing to obtain a predicted sentence vector corresponding to each of the text sentences; The determination module is used to determine whether the sentence vector generation model converges according to the predicted sentence vector and the labeled sentence vector corresponding to each of the text sentences; The generation module is used to return to the step of looping and selecting a sample data from the sample data set as the target sample data if the sentence vector generation model has not converged, until the sentence vector generation model converges, and determine the latest sentence vector generation model as the target sentence vector generation model.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the model training method according to any one of claims 1 to 6 are implemented, or the steps of the sentence vector generation method according to claim 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by the processor, the steps of the model training method according to any one of claims 1 to 6 are implemented, or the steps of the sentence vector generation method according to claim 7 are implemented.

Images