Sentence rewriting method, device, system and storage medium

Abstract

The present application provides a sentence rewriting method, apparatus, system, and storage medium. The sentence rewriting method comprises: obtaining at least one entity noun from a historical sentence based on a target entity recognition model; obtaining time information for each entity noun in the at least one entity noun; and inputting the sentence to be rewritten, the entity noun, and the time information into a target rewriting model to obtain a rewritten sentence output by the target rewriting model. Some embodiments of the present application can improve sentences to be rewritten based on entity nouns extracted from historical texts and the time at which the entity nouns were generated, effectively reducing rewriting time while improving rewriting efficiency and quality.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and more specifically, to a sentence rewriting method, device, system, and storage medium. Background Art

[0002] In intelligent systems, multi-round interactions are widely used in business scenarios. This refers to a human-machine dialogue in which the machine system initially determines the user's intent, analyzes the received input, and ultimately determines the user's instructions.

[0003] In everyday situations, users often replace unnecessary words with demonstrative pronouns or simply omit them. To help intelligent systems accurately understand user intent, existing techniques concatenate the context sentence and the sentence to be rewritten before inputting it into a model for rewriting. However, when the context sentence is long, the model processing time increases significantly, making practical application more difficult. Furthermore, directly inputting noisy context sentences into the model makes model training more difficult and time-consuming.

[0004] Therefore, how to provide an efficient statement rewriting method has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a sentence rewriting method, device, system and storage medium. Through some embodiments of the present application, the entity nouns extracted from the historical context, the time information of the entity nouns and the sentence to be rewritten can be input into the rewriting model to obtain the rewritten sentence, which effectively reduces the rewriting time while improving the rewriting efficiency and rewriting quality.

[0006] In a first aspect, some embodiments of the present application provide a sentence rewriting method, comprising: obtaining at least one entity noun in a historical sentence according to a target entity recognition model; obtaining time information of each entity noun in the at least one entity noun; inputting the sentence to be rewritten, the entity noun and the time information into a target rewriting model, and obtaining a rewritten sentence output by the target rewriting model; wherein the historical sentence is one or more sentences located before the sentence to be rewritten.

[0007] The embodiment of the present application obtains a rewritten sentence by inputting the entity nouns of the historical sentences obtained by the target entity recognition model, the time of obtaining the entity nouns, and the sentence to be rewritten into the target rewriting model. Compared with the technical solution of the related art that requires the historical sentences and the sentence to be rewritten to be spliced and then input into the model for sentence rewriting, the training process of the rewriting model is faster (this is because compared with the historical sentences input when training the rewriting model in the related art, the entity nouns in the sentence input are shorter), and the accuracy of rewriting the sentence using the rewriting model is higher (this is because compared with the related technical solution, the accuracy of the present application is higher due to the addition of the time information of the entity noun when rewriting the sentence).

[0008] In some embodiments, the target entity recognition model is trained by the following method: preprocessing the obtained original historical sentences to obtain preprocessed data, wherein the preprocessing includes removing noise in the original historical sentences and / or segmenting the sentences in the original historical sentences whose length is greater than a set threshold; dividing the preprocessed data into a first training data set and a first verification data set; training the constructed initial entity recognition model based on the data in the first training data set to obtain predicted entity nouns and an entity recognition model to be verified; and confirming that the entity recognition model to be verified has passed verification based on the first verification data set, thereby obtaining the target entity recognition model.

[0009] The present embodiment trains and verifies an initial entity recognition model to obtain a target entity recognition model. Prior to training, the acquired historical sentences are de-noised, reducing the length of the input text and the difficulty and time of model training. During sentence rewriting, the model is able to capture entity nouns that require refinement, ensuring the quality of the final sentence rewriting.

[0010] In some embodiments, the target rewriting model is obtained by training by the following method: the predicted entity noun, the time information of the predicted entity noun and the sentence to be rewritten included in the second training data are input into the rewriting model to be trained, the model to be rewritten is trained to obtain the rewriting model to be verified; according to the second verification data set, it is confirmed that the rewriting model to be verified has passed the verification, and then the target rewriting model is obtained.

[0011] This embodiment of the present application trains and verifies the initial target rewriting model using the predicted entity nouns and their time information, using the second training data. This method employs an end-to-end training approach for both models, resulting in a target rewriting model that improves sentence rewriting accuracy while reducing rewriting time.

[0012] In some embodiments, there are multiple entity nouns, wherein the rewriting method further includes: encoding the multiple entity nouns separately to obtain multiple different entity noun coding tags, wherein one entity noun corresponds to one entity noun coding tag; inputting the sentence to be rewritten, the entity noun and the time information into the target rewriting model to obtain the rewritten sentence output by the target rewriting model, including: splitting the sentence to be rewritten and performing a to-be-rewritten coding tag on each object obtained by the split to obtain a to-be-rewritten coding tag sequence; screening at least one target entity noun from the multiple entity nouns according to the time information, and obtaining the entity noun coding tags corresponding to all target entity nouns; obtaining the insertion position and / or replacement position of the at least one entity noun in the sentence to be rewritten; inserting the entity noun coding tag corresponding to the target entity noun into the insertion position and / or replacement position included in the to-be-rewritten coding tag sequence to obtain a rewriting tag sequence; and outputting the rewriting tag sequence.

[0013] The embodiment of the present application combines the sentence to be rewritten, the entity noun and the time information of the entity noun to obtain the insertion position and / or replacement position of the entity noun in the sentence to be rewritten, and finally outputs a sentence rewriting tag sequence. This method can extract the entity noun and time information, accurately locate the position that needs to be rewritten, and has high rewriting quality.

[0014] In some embodiments, splitting the sentence to be rewritten includes splitting the sentence to be rewritten using Chinese characters as splitting units.

[0015] The embodiment of the present application can accurately locate the subsequent rewriting position by splitting it into a single font.

[0016] In some embodiments, the entity recognition model to be verified and the rewriting model to be verified are obtained by confirming the following loss function:

[0017]

[0018] Among them, L is the loss function, k is the number of entity sample categories of the entity recognition model to be verified, is the label classification value of the i-th entity sample, is the probability that the output of the entity recognition model to be verified is the i-th type entity sample, n is the number of sample classification labels of the rewritten model to be verified, is the label classification value of the j-th sample classification label, The output of the rewritten model to be verified is the probability of the j-th sample classification label.

[0019] The embodiment of the present application uses a loss function to confirm whether the model training can be terminated, which is beneficial to improving the accuracy of the model.

[0020] In the second aspect, some embodiments of the present application provide a data processing method, which can achieve: semantic understanding, question retrieval or emotion recognition of the rewritten sentences obtained by the method of any embodiment in the first aspect, and obtain semantic understanding results, question retrieval results or emotion recognition results respectively.

[0021] In a third aspect, some embodiments of the present application provide a sentence rewriting device, the rewriting device comprising: an entity noun recognition module, configured to obtain at least one entity noun in a historical sentence according to a target entity recognition model; an entity noun time acquisition module, configured to obtain time information of each entity noun in the at least one entity noun; a rewriting module, configured to input the sentence to be rewritten, the entity noun and the time information into a target rewriting model, and obtain a rewritten sentence output by the target rewriting model; wherein the historical sentence is one or more sentences located before the sentence to be rewritten.

[0022] In a fourth aspect, some embodiments of the present application provide a computer device comprising one or more computers and one or more storage devices storing instructions, wherein when the instructions are executed by the one or more computers, the one or more computers perform the operations of the corresponding methods described in any embodiment of the first and second aspects.

[0023] In a fifth aspect, some embodiments of the present application provide one or more computer storage media storing instructions, which, when executed by one or more computers, enable the one or more computers to perform the operations of the corresponding methods described in any embodiment of the first and second aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A flow chart of a method for training a target entity recognition model provided in an embodiment of the present application;

[0026] Figure 2 A flow chart of a training method for a target rewriting model provided in an embodiment of the present application;

[0027] Figure 3A flow chart of a method for obtaining a target entity recognition model and a target rewriting model by training an entity recognition model and a rewriting model provided in an embodiment of the present application;

[0028] Figure 4 A model structure diagram for obtaining a target entity recognition model and a target rewriting model based on the Bi-LSTM+CRF model and the BERT model provided in an embodiment of the present application;

[0029] Figure 5 A flow chart of a method for rewriting a statement provided in an embodiment of the present application;

[0030] Figure 6 A structural diagram of a model for a sentence rewriting method provided in an embodiment of the present application;

[0031] Figure 7 A block diagram of the composition of the statement rewriting device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0033] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0034] In a related art example, a multi-turn dialogue rewriting model performs coreference resolution and omission completion on the current text sentence based on the historical context and the current input text sentence to produce the rewritten result. However, when the historical context text contains noise or is long, the model training becomes significantly more difficult and time-consuming.

[0035] Through the above analysis, it is not difficult to find that the accuracy of rewriting sentences using traditional rewriting methods is low, and the difficulty of training the model is also relatively large. In view of this, some embodiments of the present application input the entity nouns of the identified historical sentences and the time information of these entity nouns into the rewriting model to rewrite the sentences to be rewritten. It is precisely because these embodiments of the present application input entity nouns instead of the entire historical sentences into the rewriting model, and also obtain the time information of the entity nouns. Therefore, the rewriting results obtained using the model of the present application are more accurate. This is because historical sentences contain more interference information than entity nouns, which will inevitably reduce the rewriting accuracy. Compared with the technical solution that does not contain the time information of entity nouns, the embodiments of the present application also take into account the time information of entity nouns, so the accuracy of the rewritten sentences can be further improved.

[0036] It is understandable that in some embodiments of the present application, in order to further improve the accuracy of rewritten sentences, the entity recognition model and the rewriting model can be trained in an end-to-end manner during the model training phase, and the two models can be optimized at the same time. After the training is completed, the target entity recognition model and the target rewriting model are obtained. Afterwards, when there is a sentence to be rewritten, one or more adjacent historical sentences before the sentence to be rewritten can be first input into the target entity recognition model to identify the entity nouns, and the generation time information of each identified entity noun can be counted. Then, the entity nouns identified by the target entity recognition model, the time information of the entity nouns and the sentence to be rewritten are input into the target rewriting model together, and the sentence to be rewritten is rewritten, thereby improving the efficiency and accuracy of the rewriting.

[0037] It should be noted that some embodiments of the present application can be adapted to a variety of dialogue scenarios. For example, the method for rewriting sentences obtained in some embodiments of the present application can be applied in a human-computer dialogue scenario, in which the rewriting method of some embodiments of the present application is used to rewrite the sentences input by the user, and the rewritten sentences can provide the system with clearer and more complete sentence information. For example, the method for rewriting sentences obtained in other embodiments of the present application can be applied in a human-to-human online dialogue scenario, and the method can serve as an auxiliary assistant for the dialogue scenario. The rewritten sentences can provide the user with clear and complete sentence information, providing a reference for correctly understanding the semantics.

[0038] The following first introduces the process of training the entity recognition model and the rewriting model to obtain a target entity recognition model with entity noun recognition function and a target rewriting model with sentence rewriting function.

[0039] See attached Figure 1 , Figure 1 A flowchart of a method for training a target entity recognition model in some embodiments of the present application is shown.

[0040] In some embodiments of the present application, the training method of the target entity recognition model may include: S110, preprocessing the obtained original historical sentences to obtain preprocessed data, wherein the preprocessing includes removing noise in the original historical sentences and / or segmenting the sentences in the original historical sentences whose length is greater than a set threshold. S120, dividing the preprocessed data into a first training data set and a first verification data set. S130, training the constructed initial entity recognition model based on the data in the first training data set to obtain a predicted entity noun and an entity recognition model to be verified. S140, confirming that the entity recognition model to be verified has passed the verification based on the first verification data set, and then obtaining the target entity recognition model.

[0041] It is understandable that in order to ensure that the training process can be successfully completed to obtain the target entity recognition model, it is also necessary to pre-define the entity recognition loss function. The entity recognition loss function can obtain the entity recognition loss based on the difference between the predicted entity noun and the real entity noun. When the entity recognition loss does not meet the set conditions, the parameters of the entity recognition model can be adjusted by back propagation and other methods. After training the model multiple times, the entity recognition loss obtained will reach the set threshold requirement. At this time, the training process of the entity recognition model can be terminated to obtain the target entity recognition model. In other words, S130 can also include confirming that the training process of the initial entity recognition model can be terminated based on the entity recognition loss to obtain the entity recognition model to be verified.

[0042] See attached Figure 2 , Figure 2 A flowchart of a method for training a target rewriting model in some embodiments of the present application is shown.

[0043] The training method for the target rewriting model provided in the embodiments of the present application may include: S210, inputting the predicted entity noun, the time information of the predicted entity noun, and the sentence to be rewritten included in the second training data obtained in S130 into the rewriting model to be trained, training the model to be rewritten, and obtaining a rewriting model to be verified. S220, confirming that the rewriting model to be verified passes verification based on the second verification data set, thereby obtaining the target rewriting model.

[0044] It is understandable that in order to ensure that the training process can be successfully completed to obtain the target rewriting model, it is also necessary to predefine a loss function, which can obtain the rewriting loss based on the difference between the predicted rewriting statement and the actual rewriting statement. When the rewriting loss does not meet the set conditions, the parameters of the rewriting model can be adjusted by back propagation and other methods. After the rewriting model is trained multiple times, the rewriting loss obtained will reach the set threshold requirement. At this time, the training process of the rewriting model can be terminated to obtain the target rewriting model. That is to say, in some embodiments of the present application, in order to ensure the rewriting quality of the trained model. S210 can also include confirming that the training process of the initial rewriting model can be terminated based on the cross entropy of the classification task loss to obtain the rewriting model to be verified.

[0045] In some embodiments of the present application, the entity recognition loss in S130 and the classification task loss in S210 can be confirmed by the following loss function to obtain the entity recognition model to be verified and the rewriting model to be verified, and obtain a parameter model adjusted according to the loss value:

[0046]

[0047] Among them, L is the loss function, k is the number of entity sample categories of the entity recognition model to be verified, is the label classification value of the i-th entity sample, is the probability that the output of the entity recognition model to be verified is the i-th type entity sample, n is the number of sample classification labels of the rewritten model to be verified, is the label classification value of the j-th sample classification label, The output of the rewritten model to be verified is the probability of the j-th sample classification label.

[0048] Among them, if the labeling result of the sample is i, then The value of is 1, otherwise it is 0. If the labeling result of the sample is j, then The value is 1, otherwise it is 0.

[0049] The following uses the Bi-LSTM+CRF model and the BERT model as examples to explain the specific process of model training.

[0050] like Figure 3 As shown, some embodiments of the present application provide a method for training an entity recognition model and a rewriting model to obtain a target entity recognition model and a target rewriting model.

[0051] Figure 3 The methods include:

[0052] S310, collecting historical text data.

[0053] Collect original historical sentence data, manually annotated entity nouns, sentences to be rewritten, and rewritten annotated sentences from business-related system logs.

[0054] S320, historical text data preprocessing.

[0055] The obtained original historical sentences are preprocessed to obtain preprocessed data, wherein the preprocessing includes removing noise in the original historical sentences and / or segmenting sentences in the original historical sentences whose length is greater than a set threshold.

[0056] For example, since the original historical sentences collected by S310 contain meaningless special characters, spaces, and garbled characters, S320 can use regular expressions to clean up these noises. If the text length of the original historical sentences collected by S310 exceeds a set threshold, S320 uses a Python script with a segmentation function to truncate them.

[0057] S330, preparation of dataset.

[0058] The data processed by S320 is divided into a training data set and a validation data set according to a set ratio, which are used for model training and validation respectively.

[0059] As an example, in a human-computer system dialogue, both the training dataset and the validation dataset contain multiple sample data. Each sample data includes a sentence to be rewritten, at least one historical sentence, at least one entity noun, and a rewritten annotation sentence.

[0060] S340, training model.

[0061] The entity recognition model and the rewriting model are trained using the training data set obtained in step S330 to obtain a to-be-verified entity recognition model and a to-be-verified rewriting model. Then, the to-be-verified entity recognition model and the to-be-verified rewriting model are verified using the verification data set to obtain a target entity recognition model and a target rewriting model.

[0062] As an example, see the attached Figure 4 . Figure 4 A model structure diagram for obtaining a target entity recognition model and a target rewriting model based on a Bi-LSTM+CRF (Long Short-Term Memory+Conditional Random Field) model (as a specific example of an entity recognition model) and a language representation model (Bidirectional Encoder Representations from Transformers, BERT for short, as an example of a rewriting model structure) provided for some embodiments of this application. The following describes the specific training process using a training sample in human-computer interaction as an example.

[0063] first step,

[0064] Convert historical sentences into vector form and input them into the Bi-LSTM+CRF model to be trained. This process exemplarily includes: converting the text information corresponding to the historical sentences into a computer-readable vector form by splitting them into Chinese characters. Figure 4 As shown, the historical sentences include the above question and the above system response, where the above question is: "Consult Sui e Bao", and the above system response is: "It has been upgraded to Sunshine I Bao". The corresponding converted input vectors are: The vectors are represented by "E 咨 、E 询 、E 随 、E e 、E 保 ”, “E 已 、E 经 、E 升 、E 级 、E 成 、E 阳 、E 光 、E i 、E 保 ”.

[0065] Convert the sentence to be rewritten into a vector form and input it into the rewriting model to be trained (i.e. Figure 4 BERT). Figure 4 As shown, the input sentence to be rewritten is a question: "What are the advantages?", and the sentence is converted into a computer-readable vector form by splitting it into Chinese characters, and the "E 有 、E 什 、E 么 、E 优 、E 点 ”.

[0066] The second step is to obtain the entity encoding vector with time series information output by the Bi-LSTM+CRF model, namely Named Entity Recognition (NER): BP (for Sui e Bao) and IP (for Yangguang I Bao), thereby obtaining two entity nouns, and marking the entity nouns to obtain E entity1 (ie BP: with e protection) and E entity2 (i.e. IP: Sunshine IP), and then perform entity noun coding mark according to the position, i.e. E1 and E2, and finally form the entity noun coding sequence E 11 It is not difficult to understand that the appearance time BP (standing for Sui e Bao) of these two entity nouns is earlier than IP (standing for Yangguang I Bao), that is, when the appearance time information of these two entity nouns is t1 and t2 respectively, the time corresponding to t1 is earlier than the time corresponding to t2.

[0067] The third step is to select the “E” word according to the order of input. 有 、E 什 、E 么 、E 优 、E 点 "Perform position encoding on each Chinese character, where CLS indicates the encoding mark starting from E1 and SEP is the end mark, resulting in E1, E2, E3, E4, E5, E6, and E7. Then, the sentence is split and each object obtained by the split is marked with the code to be rewritten, resulting in the sequence of code marks to be rewritten E0.

[0068] The fourth step is to rewrite the coding mark sequence E0 and the time sequence information of the entity noun obtained in the second step and the entity noun coding sequence E 11 Input to the BERT model (i.e. as an example of the rewriting model).

[0069] The fifth step is to obtain the rewritten token sequence output by the BERT model. As can be seen from the figure, the entity noun encoding sequence E 11Insert the entity noun with the entity noun coding label E2 into the front of the word "have" (this is because the appearance time of this entity noun is closer to the sentence to be rewritten that is to be replaced), and obtain the predicted rewritten sentence "What advantages does sunlight I have?".

[0070] In the sixth step, analyze the cross-entropy of the classification task loss based on the predicted rewritten sentence output by the BERT model and the rewritten annotation sentences in the training dataset. If it is determined that the value of the cross-entropy is less than the set threshold, adjust the parameters of the entity recognition model and the rewriting model, and repeat the above training process.

[0071] In the seventh step, confirm that the training process of the BERT model and the entity recognition model can end based on the cross-entropy of the classification task loss, and obtain the to-be-verified BERT model and the to-be-verified entity recognition model. Then, use the verification dataset to confirm that the to-be-verified BERT model passes the verification and confirm that the entity recognition model passes the verification, and then obtain the target BERT model and the target entity recognition model.

[0072] It can be understood that in order to utilize the target entity recognition model and the target rewriting model obtained through training later, it is necessary to save the model parameters after the training ends.

[0073] The following exemplarily elaborates the specific process of the sentence rewriting method provided by the embodiments of the present application in combination with the obtained target entity recognition model and target rewriting model.

[0074] Please refer to Figure 5 , Figure 5 which is the flowchart of the sentence rewriting method provided by the embodiments of the present application.

[0075] The sentence rewriting method provided by the embodiments of the present application may include: S510, obtaining at least one entity noun in the historical sentence according to the target entity recognition model; S520, obtaining the time information of each entity noun in the at least one entity noun; S530, inputting the sentence to be rewritten, the entity noun, and the time information into the target rewriting model, and obtaining the rewritten sentence output by the target rewriting model; wherein, the historical sentence is one sentence or multiple sentences before the sentence to be rewritten.

[0076] The following exemplarily elaborates the above process.

[0077] The target entity recognition model and the target rewriting model involved in S510 are obtained through the above Figure 1 , Figure 2 or Figure 4 way of training. Some embodiments of the present application may also adopt a training process different from Figure 1 and Figure 2 to obtain the target entity recognition model and the target rewriting model. Figure 1 and Figure 2 The training process of Figure 2 is only used as a specific example.

[0078] It can be understood that at least one original historical statement needs to be collected in advance before executing S510. For example, in some embodiments of the present application, at least one original historical statement of S510 is collected from a relevant dialogue service system. Since the originally collected historical statements may have problems such as noise or long sentences, it is also necessary to denoise or truncate the original historical statements before executing S510. It is not difficult to understand that in some embodiments of the present application, the originally collected historical statements can also be both denoised and truncated.

[0079] In some embodiments of the present application, the historical statement is one or more statements before the statement to be rewritten. That is to say, the time when the historical statements selected in some embodiments of the present application are generated is earlier than the time of the statement to be rewritten. For example, if the generation time of the statement to be rewritten is t, then before time t, one or more historical statements are selected in the order from near to far from time t.

[0080] In order to accurately extract entity nouns and obtain the time information of entity generation, and accurately locate the position to be rewritten, some embodiments of the present application need to obtain the generation time of entity nouns. For example, in some embodiments of the present application, multiple entity nouns with time sequence information can be obtained through a target entity recognition model, and the time sequence information is the time information of entity noun generation; the time information of entity noun generation can also be obtained by reading the time sequence information cached inside the system. It should be noted that for the sake of clear explanation, this embodiment combines Figure 6 The provided structural diagram of the sentence rewriting method model is used for exemplary illustration. The sentence rewriting method S530 may further include the following steps.

[0081] First step, encode each of the multiple entity nouns in the historical statement obtained by using the target entity recognition model to obtain multiple different entity noun encoding tags, where one entity noun corresponds to one entity noun encoding tag.

[0082] For example, select two historical statements closest to the statement to be rewritten, input the historical statements into the trained target Bi-LSTM+CRF model (as a specific example of the target entity recognition model) above, and obtain two entity nouns. That is Figure 6 the E shown entity1 and E entity2 respectively represent the entity nouns "Sui e Bao" and "Sunshine I Bao". The entity encoding of "Sui e Bao" is marked as E1, and the entity encoding of "Sunshine I Bao" is marked as E2. Then the two entity nouns are combined into an entity encoding sequence E 11.

[0083] In the second step, the statement to be rewritten is split and each object obtained by the split is marked with a code to be rewritten to obtain a sequence of code marks to be rewritten.

[0084] For example, the sentence to be rewritten is "What are the advantages". Figure 6 As shown, these five characters are split and marked with the code to be rewritten, i.e., position codes: D1 (corresponding to "CLS", the beginning of the code), D2 (corresponding to "you"), D3 (corresponding to "shi"), D4 (corresponding to "me"), D5 (corresponding to "you"), D6 (corresponding to "dot"), and D7 (corresponding to "SEP", the end of the code). The position code data is then combined into the code mark sequence to be rewritten E0.

[0085] In the third step, at least one target entity noun is selected from the plurality of entity nouns according to the time information, and entity noun encoding tags corresponding to all target entity nouns are obtained.

[0086] For example, Figure 6 Shown are E1 and E2 in the first step.

[0087] The fourth step is to obtain the insertion position and / or replacement position of at least one entity noun in the sentence to be rewritten. The entity noun encoding token corresponding to the target entity noun is inserted into the insertion position and / or replacement position included in the encoding token sequence to be rewritten to obtain a rewriting token sequence.

[0088] For example, the coding mark sequence to be rewritten E0 and the entity coding sequence E 11 Input to the target BERT model trained above (as a specific example of a target rewriting model), Figure 6 It can be seen that the target BERT model outputs the entity encoding sequence E 11 The entity noun with entity encoding tag E2 in is inserted in front of the entity noun with position encoding D2.

[0089] Step 5: Output the rewriting mark sequence.

[0090] For example, by Figure 6 It can be seen that the rewritten sentence output by the final target BERT model is "What advantages does Sunshine I have?"

[0091] It should be noted that the target entity recognition model and the target rewriting model can also be trained by models with language recognition and processing functions other than the Bi-LSTM+CRF model and the BERT model.

[0092] As can be seen from the above, in some embodiments of the present application, the target entity recognition model and the target rewriting model obtained through training can be used in conjunction to rewrite the sentence to be rewritten based on the historical sentence. In addition, the target entity recognition model and the target rewriting model divide entity recognition and rewriting into two parts. Therefore, in other embodiments of the present application, the target entity recognition model and the target rewriting model can also be used separately.

[0093] In addition, the present invention also provides a data processing method, by executing the data processing method, it is possible to achieve: Figure 5 The rewritten sentence obtained by the method of any embodiment is subjected to semantic understanding, question retrieval or emotion recognition to obtain a semantic understanding result, a question retrieval result or an emotion recognition result respectively.

[0094] From the above, we can see that the rewritten sentence expresses more complete and clear content, making it easier for the system or people to obtain important information in terms of semantic understanding, question retrieval, or emotion recognition.

[0095] Please refer to the attached Figure 7 , Figure 7 The block diagram of the sentence rewriting device provided in the embodiment of the present application is shown. It should be understood that the sentence rewriting device is similar to the above-mentioned Figure 5 Corresponding to the method embodiment, the various steps involved in the above method embodiment can be executed. The specific functions of the statement rewriting device can be found in the description above. To avoid repetition, the detailed description is appropriately omitted here.

[0096] Figure 7 The sentence rewriting device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the sentence rewriting device. The rewriting device includes: an entity noun recognition module 710, an entity noun time acquisition module 720 and a rewriting module 730.

[0097] The entity noun recognition module 710 can be configured to obtain at least one entity noun from a historical sentence based on a target entity recognition model. The entity noun time acquisition module 720 can be configured to obtain time information for each entity noun in the at least one entity noun. The rewriting module 730 can be configured to input the sentence to be rewritten, the entity noun, and the time information into a target rewriting model, and obtain a rewritten sentence output by the target rewriting model; wherein the historical sentence is one or more sentences preceding the sentence to be rewritten.

[0098] In some embodiments of the present application, Figure 7The sentence rewriting device may also include a first training module and a second training module (not shown in the figure), wherein the first training module may be configured to: preprocess the acquired original historical sentences to obtain preprocessed data, wherein the preprocessing includes removing noise in the original historical sentences and / or segmenting the sentences in the original historical sentences whose length is greater than a set threshold; dividing the preprocessed data into a first training data set and a first verification data set; training the constructed initial entity recognition model according to the data in the first training data set to obtain predicted entity nouns and an entity recognition model to be verified; and confirming that the entity recognition model to be verified passes the verification according to the first verification data set, thereby obtaining the target entity recognition model.

[0099] The second training module can be configured to: input the predicted entity noun, the time information of the predicted entity noun and the sentence to be rewritten included in the second training data into the rewriting model to be trained, train the model to be rewritten, and obtain the rewriting model to be verified; according to the second verification data set, confirm that the rewriting model to be verified passes the verification, and then obtain the target rewriting model.

[0100] In some embodiments of the present application, the rewriting module 730 can also be configured to: encode the multiple entity nouns separately to obtain multiple different entity noun coding tags, wherein one entity noun corresponds to one entity noun coding tag; the number of the entity nouns is multiple. Split the sentence to be rewritten and perform a rewriting coding tag for each object obtained by the split (that is, split the sentence to be rewritten with Chinese characters as the splitting unit) to obtain a sequence of coding tags to be rewritten. Filter out at least one target entity noun from the multiple entity nouns based on the time information, and obtain the entity noun coding tags corresponding to all target entity nouns. Obtain the insertion position and / or replacement position of the at least one entity noun in the sentence to be rewritten. Insert the entity noun coding tag corresponding to the target entity noun into the insertion position and / or replacement position included in the sequence of coding tags to be rewritten to obtain a rewriting tag sequence. Output the rewriting tag sequence.

[0101] In some embodiments of the present application, the first training module or the second training module may further be configured to: obtain the entity recognition model to be verified and the rewriting model to be verified by confirming the following loss function:

[0102]

[0103] Among them, L is the loss function, k is the number of entity categories of the entity recognition model to be verified, is the label classification value of the i-th entity of the entity recognition model to be verified, is the probability that the entity recognition model to be verified outputs the i-th type of entity, n is the number of classification labels of the rewritten model to be verified, is the label classification value of the j-th classification label of the rewritten model to be verified, Rewrite the model output to be verified as the probability of the j-th classification label.

[0104] Some embodiments of the present application further provide a computer device comprising one or more computers and one or more storage devices storing instructions, wherein when the instructions are executed by the one or more computers, the one or more computers execute Figure 5 The operations of the method of any embodiment.

[0105] Some embodiments of the present application also provide one or more computer storage media storing instructions, which, when executed by one or more computers, cause the one or more computers to execute Figure 5 The operations of the method of any embodiment.

[0106] The foregoing is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0107] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0108] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

Claims

1. A method for rewriting a sentence, characterized in that: The rewriting method comprises: Obtain at least one entity noun in the historical sentences according to the target entity recognition model; Obtaining time information of each entity noun in the at least one entity noun; Inputting the sentence to be rewritten, the entity noun and the time information into the target rewriting model, and obtaining a rewritten sentence output by the target rewriting model; wherein the historical sentence is one or more sentences located before the sentence to be rewritten; The target entity recognition model and the target rewriting model are obtained by adopting an end-to-end training method; There are multiple entity nouns, wherein the rewriting method further comprises: encoding the multiple entity nouns respectively to obtain multiple different entity noun encoding tags, wherein one entity noun corresponds to one entity noun encoding tag; The step of inputting the sentence to be rewritten, the entity noun, and the time information into a target rewriting model and obtaining a rewritten sentence output by the target rewriting model includes: The sentence to be rewritten is split into Chinese characters as the splitting unit, and each object obtained by the split is marked with a code to be rewritten to obtain a code mark sequence to be rewritten; at least one target entity noun is filtered out from the multiple entity nouns according to the time information, and the entity noun code marks corresponding to all target entity nouns are obtained; the insertion position and / or replacement position of the at least one entity noun in the sentence to be rewritten is obtained; the entity noun code mark corresponding to the target entity noun is inserted into the insertion position and / or replacement position included in the code mark sequence to be rewritten to obtain a rewriting mark sequence; and the rewriting mark sequence is output.

2. The rewriting method according to claim 1, wherein: The target entity recognition model is trained by the following method: Preprocessing the acquired original historical sentences to obtain preprocessed data, wherein the preprocessing includes removing noise from the original historical sentences and / or segmenting sentences in the original historical sentences whose length is greater than a set threshold; Dividing the preprocessed data into a first training data set and a first validation data set; Training the constructed initial entity recognition model based on the data in the first training data set to obtain a predicted entity noun and an entity recognition model to be verified; According to the first verification data set, it is confirmed that the entity recognition model to be verified passes the verification, and then the target entity recognition model is obtained.

3. The rewriting method according to claim 2, wherein: The target rewriting model is trained by the following method: Inputting the predicted entity noun, the time information of the predicted entity noun, and the sentence to be rewritten included in the second training data into a rewriting model to be trained, training the model to be rewritten, and obtaining a rewriting model to be verified; According to the second verification data set, it is confirmed that the rewriting model to be verified passes the verification, and then the target rewriting model is obtained.

4. The method according to any one of claims 1 to 3, characterized in that The entity recognition model to be verified and the rewriting model to be verified are obtained by confirming the following loss function: Among them, L is the loss function, k is the number of entity sample categories of the entity recognition model to be verified, is the label classification value of the i-th entity sample, is the probability that the output of the entity recognition model to be verified is the i-th type entity sample, n is the number of sample classification labels of the rewritten model to be verified, is the label classification value of the j-th sample classification label, The output of the rewritten model to be verified is the probability of the j-th sample classification label.

5. A data processing method, characterized in that: The data processing method is executed to achieve: semantic understanding, question retrieval or emotion recognition of the rewritten sentence obtained according to any one of the methods of claims 1-4, and obtain semantic understanding results, question retrieval results or emotion recognition results respectively.

6. A sentence rewriting device, characterized in that: The rewriting device is used to execute the method according to claim 1, comprising: an entity noun recognition module, configured to obtain at least one entity noun in a historical sentence according to a target entity recognition model; an entity noun time acquisition module, configured to acquire time information of each entity noun in the at least one entity noun; a rewriting module configured to input the sentence to be rewritten, the entity noun, and the time information into a target rewriting model, and obtain a rewritten sentence output by the target rewriting model; The historical statement is one or more statements preceding the statement to be rewritten.

7. A computer device comprising one or more computers and one or more storage devices storing instructions, which, when executed by the one or more computers, cause the one or more computers to perform the operations of the method according to any one of claims 1 to 5.

8. One or more computer storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform the operations of the method according to any one of claims 1-5.

Images