A similar sentence generation method and device, computing device, and storage medium

Abstract

The application discloses a similar sentence generation method applied to the field of data processing and used for improving stability and accuracy of similar sentence generation. The method comprises the following steps: obtaining an original sentence, inputting the original sentence into an operation sequence generation model, obtaining a target operation sequence output by the operation sequence generation model, and operating the original sentence according to an operation mode indicated by the target operation sequence to obtain a similar sentence corresponding to the original sentence. In the application, the output of the operation sequence generation model is a sequence form capable of indicating the operation mode of the similar sentence generation. Since the data amount of the sequence form is small, the model needs to process less information, and therefore the sequence form can be stably and accurately generated. Then, the similar sentence can be generated by performing corresponding operations on the original sentence according to the sequence form. The similar sentence generated by using the method has a low probability of being a wrong sentence or having a different semantic meaning from the original sentence.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more particularly to a method, apparatus, computing device, and storage medium for generating similar statements. Background Technology

[0002] With the rapid development of computer science, the internet has become an important channel for users to obtain information and knowledge daily. However, due to the rapid increase in data on the internet, it is difficult for users to quickly obtain the effective information they need from search engines. The emergence of automated question-and-answer systems has solved this problem. An automated question-and-answer system is a system that can automatically interact with users, allowing them to ask questions in natural language and directly return the answers they need, which is convenient and efficient.

[0003] A core challenge for automated question-answering systems is improving their natural language understanding of user-input questions. Typically, this can be achieved by collecting a large dataset of similar statements to train a natural language understanding model, thereby enhancing its comprehension and generalization abilities. Therefore, it can be said that the accuracy and stability of generating similar questions determine the accuracy and stability of the automated question-answering system. However, current methods for generating similar questions exhibit poor stability and accuracy.

[0004] In summary, there is a current need for a similar statement generation method to improve the stability and accuracy of generating similar questions. Summary of the Invention

[0005] This invention provides a method, apparatus, computing device, and storage medium for generating similar statements. By inputting the original statement into an operation sequence generation model to obtain a target operation sequence, and generating similar statements corresponding to the original statement based on the target operation sequence, this invention solves the problems of low stability and poor accuracy in existing similarity problem generation methods.

[0006] In a first aspect, the present invention provides a method for generating similar statements. The method includes: first, acquiring an original statement; then, inputting the original statement into an operation sequence generation model and obtaining a target operation sequence output by the model; and finally, operating the original statement according to the operation method indicated by the target operation sequence to obtain similar statements corresponding to the original statement. The operation sequence generation model is trained based on a reference statement, similar statements corresponding to the reference statement, and a reference operation sequence used to convert the reference statement into similar statements. The reference operation sequence is used to identify the operation method used to operate the reference statement to obtain similar statements corresponding to the reference statement.

[0007] In the methods described above, the operation sequence generation model takes the original statement as input (text) and outputs a sequence indicating the specific operations for generating similar statements. Because the sequence data is relatively small, the model needs to process less information and can stably and accurately generate the target operation sequence. In contrast, existing technologies typically input the original statement into a similarity sentence generation model to directly obtain similar statements. Since this model takes text as both input and output, it requires processing more information and cannot stably and accurately generate similar statements, making the generated similar statements prone to errors or semantic differences from the original. However, this approach first stably and accurately generates the target operation sequence, and then performs operations on the original statement according to the target operation sequence, thus stably and accurately generating similar statements. The probability of similar statements generated using this method being incorrect or semantically different from the original is low.

[0008] Optionally, the operation sequence generation model is constructed as follows: M reference statements and similar statements corresponding to each of the M reference statements are obtained, where M is a positive integer; for each of the M reference statements, the operation method for converting the reference statement into each similar statement corresponding to the reference statement is determined, and a corresponding reference operation sequence is generated based on the operation method; the reference statement and its corresponding reference operation sequence are used as feature data; the generative pre-trained Transform-2 model GPT-2 is trained using all the feature data corresponding to the M reference statements to obtain the operation sequence generation model.

[0009] In this approach, the amount of data in the operation sequences is relatively small, resulting in a correspondingly smaller amount of feature data. Consequently, the dataset of similar sentences used to train the operation sequence generation model is also smaller, making it easier to collect and significantly reducing labor costs. Furthermore, using all the feature data, which is relatively small, also improves the processing complexity during operation sequence generation model training, thereby increasing training efficiency. In addition, the operation sequence generation model in this approach is the GPT-2 model, which possesses stronger language understanding capabilities, higher generation quality, and lower inference and computational costs, making this approach more feasible in practical applications.

[0010] Optionally, the operation method for converting the reference statement into each similar statement corresponding to the reference statement is determined, and a corresponding reference operation sequence is generated according to the operation method, including: performing word segmentation on the reference statement and each similar statement respectively to obtain each semantic unit of the reference statement and each semantic unit of each similar statement; for each semantic unit in the reference statement, comparing the semantic unit with each semantic unit in each similar statement to determine the operation method for the semantic unit when converting the reference statement into each similar statement; and combining the operation methods corresponding to each semantic unit according to the order of each semantic unit in the reference statement to obtain a reference operation sequence.

[0011] In this scheme, a semantic unit refers to the smallest unit with a specific meaning in a language, such as a word. By segmenting the reference statement and its corresponding similar statements before generating the operation sequence, not only can the computer better understand these languages, but it can also more effectively compare each semantic unit of the reference statement with each semantic unit of the similar statements, quickly and accurately determining what operations can be performed on the reference statement to convert it into a similar statement.

[0012] Optionally, the operation methods include one or more of the following: delete, keep, insert, and swap.

[0013] In the above method, by performing one or more of the following operations on each semantic unit of the reference statement: deletion, retention, insertion, and exchange, the reference statement can be transformed into a similar statement corresponding to the reference statement.

[0014] Optionally, after performing word segmentation on the reference statement and each similar statement, the method further includes: if it is determined that the reference statement and the similar statement contain the same punctuation mark, and the content before the punctuation mark in the reference statement is the same as the content after the punctuation mark in the similar statement, and the content after the punctuation mark in the reference statement is the same as the content before the punctuation mark in the similar statement, then the operation mode corresponding to the reference statement is determined to be swap.

[0015] In the above method, when the reference statement and its similar statement share the same punctuation mark, for example, when both the reference statement and its similar statement contain a comma, and the content before the comma in the reference statement is the same as the content after the comma in the similar statement, and the content after the comma in the reference statement is the same as the content before the comma in the similar statement, it means that the similar statement is obtained by swapping the short sentences of the reference statement. The similar statement can be obtained by exchanging the content after the comma in the reference statement with the content before the comma. Therefore, the operation method corresponding to the reference statement in the above situation is determined to be swapping.

[0016] Optionally, determining the operation mode for semantic units when converting the reference statement into each similar statement includes: sequentially traversing each semantic unit of the reference statement, and for each semantic unit currently traversed, determining that the operation mode for the semantic unit is to be retained if the semantic unit is the same as the semantic unit in the similar statement.

[0017] In the above method, if a semantic unit of a reference statement is the same as a semantic unit of a similar statement, then when converting the reference statement into its similar statement, the semantic unit can be retained, and the operation mode of the semantic unit is marked as retained.

[0018] Optionally, for each semantic unit in the reference statement, before comparing the semantic unit with each semantic unit in each similar statement, the method further includes: obtaining candidate semantic units in each similar statement that are different from the corresponding reference statement; then, determining the frequency of each candidate semantic unit in all candidate semantic units; when the frequency is greater than a first threshold, adding the candidate semantic unit to the phrase set; after determining that the operation mode of the semantic unit is to retain it, the method further includes: if there are semantic units in the similar statement that do not exist in the reference statement before the semantic unit, then searching the phrase set based on the semantic units that do not exist in the reference statement; when there are semantic units in the phrase set that do not exist in the reference statement, determining that the operation mode of the semantic units that do not exist in the reference statement is to insert them; when there are no semantic units in the phrase set that do not exist in the reference statement, determining that the operation mode of the semantic units that do not exist in the reference statement is to delete them.

[0019] In the above method, candidate semantic units with a frequency greater than a first threshold can be considered as relatively common and frequently used semantic units. Thus, by constructing a phrase set from candidate semantic units in each similar sentence with a frequency greater than the first threshold, it is possible to construct a phrase set from candidate semantic units that appear frequently in similar sentences, making the phrase set representative. Then, by comparing each semantic unit in the reference sentence with each semantic unit in each similar sentence, the operation method for each semantic unit in the similar sentences can be determined.

[0020] Optionally, the original statement is operated according to the operation mode indicated by the target operation sequence to obtain similar statements corresponding to the original statement, including: when the operation mode indicated by the target operation sequence includes insertion, one or more candidate semantic units are selected from the phrase set and inserted into the original statement, wherein the total length of the one or more candidate semantic units is less than a second threshold.

[0021] In the above method, when selecting candidate semantic units from the phrase set to insert into the original sentence, by setting the total length of the candidate semantic units to be lower than the second threshold, the problem of excessively long similar sentences generated due to inserting too many candidate semantic units into the original sentence can be avoided.

[0022] Secondly, the present invention provides a similar statement generation apparatus, comprising: an acquisition unit for acquiring an original statement; and a processing unit for inputting the original statement into an operation sequence generation model to obtain a target operation sequence output by the operation sequence generation model; wherein the operation sequence generation model is trained based on a reference statement, similar statements corresponding to the reference statement, and a reference operation sequence used to convert the reference statement into the similar statement, the reference operation sequence being used to identify the operation method of operating the reference statement to obtain the similar statement corresponding to the reference statement; and the original statement is operated according to the operation method indicated by the target operation sequence to obtain the similar statement corresponding to the original statement.

[0023] Optionally, the processing unit is specifically used to obtain M reference statements and similar statements corresponding to each of the M reference statements, where M is a positive integer; for each of the M reference statements, determine the operation method to convert the reference statement into each similar statement corresponding to the reference statement, and generate a corresponding reference operation sequence according to the operation method, and use the reference statement and the corresponding reference operation sequence as feature data; use all the feature data corresponding to the M reference statements to train a generative pre-trained Transform-2 model to obtain an operation sequence generation model.

[0024] Optionally, the processing unit is specifically configured to perform word segmentation on the reference statement and each similar statement respectively, to obtain each semantic unit of the reference statement and each semantic unit of each similar statement; for each semantic unit in the reference statement, compare the semantic unit with each semantic unit in each similar statement to determine the operation mode of the semantic unit when converting the reference statement into each similar statement; and combine the operation modes corresponding to each semantic unit according to the order of the semantic units in the reference statement to obtain the reference operation sequence.

[0025] Optionally, the operation includes one or more of the following: deletion, retention, insertion, and swapping.

[0026] Optionally, the processing unit is specifically configured to determine that the operation mode corresponding to the reference statement is swap if it is determined that the reference statement and the similar statement contain the same punctuation mark, and the content of the reference statement before the punctuation mark is the same as the content of the similar statement after the punctuation mark, and the content of the reference statement after the punctuation mark is the same as the content of the similar statement before the punctuation mark, then the operation mode corresponding to the reference statement is swap.

[0027] Optionally, the processing unit is specifically used to traverse each semantic unit of the reference statement in sequence, and for each semantic unit currently traversed, if it is determined that the semantic unit is the same as a semantic unit in a similar statement, the operation mode of the semantic unit is determined to be retained.

[0028] Optionally, the processing unit is specifically used to obtain candidate semantic units that are different from the corresponding reference statement in each similar statement; determine the frequency of each candidate semantic unit in all candidate semantic units, and when the frequency is greater than a first threshold, add the candidate semantic unit to the phrase set; after determining that the operation mode of the semantic unit is to retain it, it further includes: if there are semantic units in the similar statements that do not exist in the reference statement before the semantic unit, then search the phrase set according to the semantic units that do not exist in the reference statement; when there are semantic units in the phrase set that do not exist in the reference statement, determine that the operation mode of the semantic units that do not exist in the reference statement is to insert them; when there are no semantic units in the phrase set that do not exist in the reference statement, determine that the operation mode of the semantic units that do not exist in the reference statement is to delete them.

[0029] Optionally, the processing unit is specifically configured to select one or more candidate semantic units from the phrase set and insert them into the original statement when the operation mode indicated by the target operation sequence includes insertion, wherein the total length of the one or more candidate semantic units is less than a second threshold.

[0030] Thirdly, the present invention provides a computing device including at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the similar statement generation method described in any of the first aspects above.

[0031] Fourthly, the present invention also provides a computer-readable storage medium storing a program that, when run on a computer, causes the computer to execute any of the similar statement generation methods described in the first aspect.

[0032] For details of the beneficial effects of the second to fourth aspects mentioned above, please refer to the technical effects that can be achieved by the corresponding design in the first aspect mentioned above, which will not be repeated here. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in this invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic flowchart of a method for constructing an operation sequence generation model provided by an embodiment of the present invention;

[0035] Figure 2 This is a schematic flowchart of a method for generating a reference operation sequence provided in an embodiment of the present invention;

[0036] Figure 3 This is a flowchart illustrating a method for determining a set of phrases according to an embodiment of the present invention;

[0037] Figure 4 This is a flowchart illustrating a method for determining the operation mode of other semantic units according to an embodiment of the present invention;

[0038] Figure 5 This is a schematic flowchart of a method for generating an operation sequence generation model provided by an embodiment of the present invention;

[0039] Figure 6 This is a flowchart illustrating a method for generating similar statements according to an embodiment of the present invention;

[0040] Figure 7 This is a structural diagram of a similar statement generation device provided in an embodiment of the present invention;

[0041] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0043] This invention provides a method for generating similar statements. By inputting the original statement into an operation sequence generation model to obtain a target operation sequence, and generating similar statements corresponding to the original statement based on the target operation sequence, this method solves the problems of low stability and poor accuracy of existing similar statement generation methods.

[0044] It should be noted that, in the embodiments of the present invention, before generating the original statement into the corresponding similar statement according to the operation sequence generation model, the operation sequence generation model needs to be trained first. The training process of the operation sequence generation model will be described in detail below.

[0045] Please refer to Figure 1The diagram shown is a flowchart illustrating a method for constructing an operation sequence generation model according to an embodiment of the present invention. This method is applicable to similar statement generation devices and includes the following steps:

[0046] Step 101: Obtain M reference statements and the similar statements corresponding to each of the M reference statements.

[0047] In this embodiment of the invention, constructing an operation sequence generation model requires first obtaining a dataset of similar statements used to train the model. For example, the dataset of similar statements may include M reference statements (M being a positive integer) and similar statements corresponding to each of the M reference statements. Similar statements to reference statements refer to statements that are semantically similar to the reference statements. The statements can be declarative sentences, interrogative sentences, or sentences containing multiple clauses; they can be in Chinese, English, or a mixture of both, without specific limitations.

[0048] For example, taking the reference statement "How to learn programming?" as an example, similar statements could include "How to learn programming effectively?", "What are some methods to learn programming well?", "How should programming be learned?", "What methods are there to learn programming?", and so on. Taking the reference statement "The weather is very nice today." as an example, similar statements could include "The weather is very pleasant today.", "The weather is very beautiful today.", "The sun is shining brightly today.", "Today is a sunny day.", "The weather is nice today," and so on.

[0049] It is understandable that the more similar sentences a reference sentence corresponds to, the better the similar sentence generation device learns how to convert the reference sentence into its corresponding similar sentences. Therefore, to improve the performance of the trained model, it is necessary to select reference sentences with a large number of corresponding similar sentences as training data. Based on this, in an optional implementation, a quantity threshold can be preset. When the number of similar sentences corresponding to a reference sentence is greater than the quantity threshold, the reference sentence and all its corresponding similar sentences can be collected into the similar sentence dataset. The quantity threshold can be set by those skilled in the art based on experience, or it can be determined experimentally, for example, by experimenting with various values ​​to find a value that results in good model performance. In one example, the quantity threshold can be set to 5, that is, when the number of similar sentences corresponding to a reference sentence is greater than 5, the reference sentence and all its corresponding similar sentences can be collected into the similar sentence dataset.

[0050] However, the above acquisition method is only an ideal method. Because, in actual application, it is usually necessary to obtain a corresponding similar sentence dataset for a specific business scenario. For some special business scenarios, it is relatively difficult to collect the similar sentences corresponding to the reference sentences. Therefore, the number of similar sentences corresponding to the collected reference sentences is also relatively small.

[0051] To solve this problem, in a possible implementation, the reference sentences and the similar sentences corresponding to the reference sentences can also be preprocessed to achieve sentence expansion, so as to obtain more reference sentences and more similar sentences corresponding to the reference sentences. Among them, the preprocessing can include one or more of the following processing methods:

[0052] Method 1, back translation processing, refers to the processing method of translating a sentence that has been translated into another language back into the original language. For example, taking the original language as Chinese and the translated language as English, back translation processing means translating the original Chinese language into English, and then translating the English language back into Chinese. For example, taking the original language sentence "I smell the aroma of barbecue and hear the laughter of children." as an example, the Chinese sentence can be first translated into English: "I smell the aroma of barbecue and hear the laughter of children.", and then the English sentence is translated back into Chinese: "我闻到了烤肉的香味,听到了孩子们的笑声。" It can be seen that although the sentence after back translation is different from the original language sentence, the semantics are similar. Therefore, through the back translation processing of Chinese and English, similar sentences of the original language sentence are obtained.

[0053] In addition, the back translation processing can also include translation processing between dialects. For example, the original language of Mandarin can be converted into a dialect language, and then the dialect language is translated back into Mandarin. Since the grammar and sentence patterns are different between different languages, different styles of sentences can be generated during the back translation processing.

[0054] Method two, synonym replacement, refers to replacing words in a reference sentence with synonyms. For example, easy data augmentation (EDA) can be used to randomly select several non-stop words from the sentence and replace them with their synonyms. Stop words are common words that appear in almost every document. For instance, taking the original sentence "The weather is very nice today," after synonym replacement, a similar sentence is obtained: "The weather is very pleasant today." It is evident that synonym replacement may result in incoherent sentences. Including these incoherent sentences in the operation sequence generation model can improve the diversity of the similar sentence dataset and prevent the operation sequence generation model from overfitting.

[0055] It should be noted that preprocessing methods can include other approaches besides the two mentioned above. By preprocessing the reference statement and its corresponding similar statements, more similar statements can be obtained, thereby expanding the similar statement dataset. This increases the number of samples for training the operation sequence generation model, ultimately improving the model's performance.

[0056] Step 102: For each of the M reference statements, determine the operation method to convert the reference statement into each similar statement corresponding to the reference statement, and generate the corresponding reference operation sequence according to the operation method. The reference statement and the corresponding reference operation sequence are used as feature data.

[0057] The term "operation method" refers to the actions performed when converting a reference statement into a similar statement. This operation method may include one or more of the following: deletion, retention, insertion, and swapping. In some embodiments, it may also include adding punctuation marks and / or using pronouns to replace the subject.

[0058] For example, to illustrate operations including deletion, retention, insertion, and swapping, please refer to [link to relevant documentation]. Figure 2 The diagram shown is a flowchart illustrating a method for generating a reference operation sequence according to an embodiment of the present invention. The method includes the following steps:

[0059] Step 201: Perform word segmentation on the reference statement and each similar statement respectively to obtain the semantic units of the reference statement and each semantic unit of each similar statement.

[0060] A semantic unit refers to the smallest unit in a language that has a specific meaning. A sentence can contain multiple semantic units. Since each semantic unit has its unique meaning and function, these semantic units together constitute the complete meaning of the sentence.

[0061] For example, taking the reference statement "How to learn programming?" and its corresponding similar statement "How to learn programming effectively?" as an example, the word segmentation of the reference statement "How to learn programming?" can yield three semantic units: "how", "learn", and "programming"; while the word segmentation of the corresponding similar statement "How to learn programming effectively?" can yield four semantic units: "how", "effectively", "learn", and "programming".

[0062] In an alternative implementation, statements can also be labeled as sequences. For example, a reference statement can be labeled as sequence S, and the corresponding similar statements can be labeled as sequence T.

[0063] For example, still taking the reference statement as "How to learn programming?" and its corresponding similar statement as "How to learn programming effectively?" as an example, assuming the reference statement is labeled as sequence S and the corresponding similar statement is labeled as sequence T, then:

[0064] First, the reference statements can be divided into three semantic units in sequence: "how," "learn," and "programming." Therefore, the sequence S can be written as:

[0065] Sequence S = [S[1]S[2]S[3]]

[0066] Among them, S[1] represents the semantic unit "how", S[2] represents the semantic unit "learning", and S[3] represents the semantic unit "programming".

[0067] Correspondingly, similar statements can be divided into four semantic units in sequence: "how," "effectively," "learning," and "programming." Therefore, sequence T can be written as:

[0068] Sequence T = [T[1]T[2]T[3]T[4]]

[0069] Wherein, T[1] represents the semantic unit “how”, T[2] represents the semantic unit “effectively”, T[3] represents the semantic unit “learning”, and T[4] represents the semantic unit “programming”.

[0070] Further, optionally, after performing word segmentation on the reference statement and each similar statement, it can be determined whether the reference statement and the similar statement contain the same punctuation mark. If it is determined that the reference statement and the similar statement contain the same punctuation mark, and the content before the punctuation mark in the reference statement is the same as the content after the punctuation mark in the similar statement, and the content after the punctuation mark in the reference statement is the same as the content before the punctuation mark in the similar statement, then the operation mode corresponding to the reference statement is determined to be swapping.

[0071] For example, taking the reference statement "I was watching TV when the doorbell rang." and its similar statement "When the doorbell rang, I was watching TV" as an example, the reference statement and the similar statement share the same punctuation mark, comma ",". Furthermore, the content before the punctuation mark "," in the reference statement, "I was watching TV," is the same as the content after the punctuation mark "," in the similar statement. Similarly, the content after the punctuation mark "," in the reference statement, "When the doorbell rang," is the same as the content before the punctuation mark "," in the similar statement. Therefore, it can be determined that the operation corresponding to the reference statement is swapping.

[0072] Further, optionally, after obtaining the semantic units of the reference statement and the semantic units of each similar statement, a phrase set can be constructed to represent the semantic units frequently used in similarity substitutions. For details, please refer to [link to relevant documentation]. Figure 3 The diagram shown is a flowchart illustrating a method for determining a phrase set according to an embodiment of the present invention. The method includes the following steps:

[0073] Step 301: Obtain the candidate semantic units that are different from the corresponding reference statements in each similar statement.

[0074] In the aforementioned example, the reference statement S is “How to learn programming?”, and the corresponding similar statement T is “How to learn programming effectively?”. It can be seen that the semantic unit T[3] “learn” in the similar statement T is the same as the semantic unit S[2] “learn” in the reference statement S, and the semantic unit T[4] “program” in the similar statement T is the same as the semantic unit S[3] “program” in the reference statement S. However, the semantic units T[1] “how” and T[2] “effectively” in the similar statement T are different from the semantic units of the corresponding reference statement. Therefore, the semantic units T[1] “how” and T[2] “effectively” in the similar statement T are determined as the candidate semantic units.

[0075] In one possible implementation, the longest common subsequence (LCS) algorithm can be used to first determine the longest common substring of each similar statement and its corresponding reference statement. Then, semantic units that do not belong to the longest common subsequence are determined as candidate semantic units.

[0076] Step 302: Determine the frequency of each candidate semantic unit among all candidate semantic units. When the frequency is greater than the first threshold, add the candidate semantic unit to the phrase set.

[0077] In detail, after obtaining all candidate semantic units, the frequency of each candidate semantic unit among all candidate semantic units can be determined. Candidate semantic units with a frequency greater than a first threshold can be considered relatively common and frequently used semantic units. Therefore, when the frequency is greater than the first threshold, the candidate semantic unit can be added to the phrase set. In this way, a phrase set is constructed using candidate semantic units that appear frequently in similar sentences, making the phrase set more representative.

[0078] Step 202: For each semantic unit in the reference statement, compare the semantic unit with each semantic unit in each similar statement to determine the operation method for the semantic unit when converting the reference statement into each similar statement.

[0079] Optionally, after obtaining all semantic units in the reference statement and all semantic units in the similar statements, the semantic units of the reference statement can be traversed sequentially. For each semantic unit currently traversed, it is determined whether its semantic unit is the same as the semantic units in the similar statements. If the currently traversed semantic unit is different from the semantic units in the similar statements, the semantic unit is skipped, and the traversal continues to the next semantic unit in the similar statements. Conversely, if the currently traversed semantic unit is the same as the semantic unit in the reference statement, the semantic unit is retained. The traversal order can refer to the order from the beginning to the end of the sentence or the order from the end to the beginning of the sentence; the specific order is not limited.

[0080] Taking the aforementioned example, for the reference statement S "How to learn programming?" and the similar statement T "How to learn programming effectively?", assuming that the semantic units in the reference statement S are traversed in order from the beginning to the end of the sentence, then: First, traverse the semantic unit S[1] "how", and find that there is no identical semantic unit in the similar statement T, so skip the semantic unit S[1] and continue to traverse the next semantic unit S[2] "learn"; compare and find that the semantic unit S[2] currently traversed is the same as the semantic unit T[3] "learn" in the similar statement T, so determine that the operation mode of the semantic unit S[2] currently traversed is to be retained, and then continue to traverse the next semantic unit S[3] "program"; compare and find that the semantic unit S[3] currently traversed is the same as the semantic unit T[4] "program" in the similar statement T, so determine that the operation mode of the semantic unit S[3] currently traversed is also to be retained.

[0081] Further exemplarily, following the above method, in addition to semantic units whose operation methods are already determined, there may also be other semantic units in the reference statement whose operation methods are not yet determined. Therefore, it is also necessary to determine the operation methods corresponding to these other semantic units whose operation methods are not yet determined. There are many ways to determine the operation methods of other semantic units. For example, in one possible method (called determination method one), during the process of traversing each semantic unit of the reference statement, after each semantic unit whose operation type is reserved is determined, it can be used as a benchmark to query whether there are other semantic units in the reference statement whose operation methods are not yet determined before that semantic unit. If so, the operation method of the other semantic unit is then analyzed, and after the analysis is completed, the next semantic unit in the reference statement is traversed. For example, in another possible determination method (referred to as determination method two), we can first traverse each semantic unit of the reference statement in sequence. For each semantic unit currently being traversed, if the operation type of the semantic unit is determined to be reserved, we continue to traverse the next semantic unit. After traversing all semantic units, we then traverse all semantic units in the reference statement whose operation type has been determined to be reserved. For each semantic unit whose operation type is reserved, we analyze the other semantic units whose operation type is not determined before the semantic unit whose operation type is reserved. After determining the operation type of the other semantic units, we continue to traverse the next semantic unit in the reference statement whose operation type is reserved.

[0082] Please see Figure 4 This is a flowchart illustrating a method for determining the operation mode of other semantic units according to an embodiment of the present invention. The method is described using the determination of semantic units with undetermined operation types before one of the semantic units whose operation type has been determined to be reserved as an example. The method includes the following steps:

[0083] Step 401: Determine whether there is a semantic unit in the similar statement that does not exist in the reference statement before the semantic unit. If it exists, proceed to step 402; otherwise, proceed to step 404.

[0084] Taking the aforementioned example, the reference statement S is "How to learn programming?", and the corresponding similar statement T is "How to learn programming effectively?". Assuming that other semantic units are analyzed according to the above determination method one, after finding that the semantic unit T[3] "learn" in the similar statement T is the same as the semantic unit S[2] "learn" in the reference statement S, the operation mode of the semantic unit S[2] in the reference statement S is determined to be retained. Then, it is determined whether there is a semantic unit in the similar statement T that does not exist in the reference statement S before the semantic unit T[3] (that is, T[1] and T[2]). It can be seen that the semantic unit T[1] "how" and the semantic unit T[2] "effectively" are semantic units that do not exist in the reference statement S. Therefore, there is a semantic unit in the similar statement T that does not exist in the reference statement S before the semantic unit T[3]. Step 402 is executed.

[0085] Step 402: Search the phrase set for semantic units that do not exist in the reference statement, and determine whether there are semantic units in the phrase set that do not exist in the reference statement. If they do, proceed to step 403; otherwise, proceed to step 404.

[0086] In the aforementioned example, the phrase set is searched based on the semantic units T[1] "how" and T[2] "effectively" that do not exist in the reference statement. When the phrase "how" exists in the phrase set, it means that the semantic unit T[1] "how" that does not exist in the reference statement exists in the phrase set, and step 403 can be executed. When the phrase "effectively" does not exist in the phrase set, it means that the semantic unit T[2] "effectively" that does not exist in the reference statement exists in the phrase set, and step 404 can be executed. For a related introduction to the phrase set, please refer to the aforementioned Figure 3 This will not be elaborated upon here.

[0087] Step 403: Determine that the operation mode for semantic units that do not exist in the reference statement is insertion.

[0088] In detail, when there are semantic units in the phrase set that do not exist in the reference statement compared to similar statements, the operation mode for determining these semantic units in the reference statement is insertion. For example, in the previous example, since there is a semantic unit T[1] "how" in the phrase set that does not exist in the reference statement compared to similar statements, the operation mode for the semantic unit S[1] in the reference statement is determined to be insertion, and thus the phrase "how" is inserted into the semantic unit S[1] in the reference statement.

[0089] It should be noted that for each operation that requires insertion, the phrase to be inserted can be marked using a key-value pair. For example, different characters can be used to mark different operations that require insertion, indicating which phrase in the phrase set should be inserted during insertion.

[0090] Step 404: Determine that the operation mode for non-existent semantic units is deletion.

[0091] In detail, when there are no semantic units in the phrase set that do not exist in the reference statement compared to similar statements, the operation to determine these semantic units in the reference statement that do not exist in the similar statements is deletion.

[0092] In the above implementation, by first identifying the semantic units whose operation mode is to be retained, and then using the semantic units whose operation mode is to be retained as a benchmark, analyzing the operation modes of other semantic units before them, so as to determine whether the operation mode of other semantic units belongs to deletion or insertion, the operation mode of the entire reference statement can be accurately compared with the operation mode of each semantic unit in similar statements, providing support for the subsequent generation of operation sequences.

[0093] Step 203: According to the order of each semantic unit in the reference statement, combine the operation methods corresponding to each semantic unit to obtain the reference operation sequence.

[0094] In one possible implementation, the operation mode can be assigned a value to obtain the operation value O. Taking the previous example, refer to the statement S "How to learn programming?", the corresponding similar statement T "How to learn programming effectively?", assuming that the operation mode of deletion is assigned a value of 0, the operation mode of retention is assigned a value of 1, and the operation mode of insertion is assigned a value of 2, then the operation mode of the semantic unit S[1] "how" of the reference statement is insertion, so O[1] = 2; the operation mode of the semantic unit S[2] "learn" of the reference statement is retention, so O[2] = 1; the operation mode of the semantic unit S[3] "program" of the reference statement is retention, so O[3] = 1.

[0095] It should be understood that, following the assignment method described above, other operations can be assigned corresponding values. For example, the operation of adding punctuation marks can be assigned a value of 3, and the operation of using a pronoun to replace the subject can be assigned a value of 4. Furthermore, assigning values ​​to operations in numerical form is only an example; in practice, non-numerical forms can also be used to assign values ​​to operations without limitation.

[0096] In a further implementation, an initial operation sequence can be created first based on the sequence S of the reference statement. Then, the operation values ​​corresponding to the operation methods of each semantic unit are combined according to the order of the semantic units in the reference statement to obtain the reference operation sequence. Here, the initial operation sequence means that the operation value of each semantic unit in the operation sequence is assigned to 0. That is, for the initial operation sequence O, the default operation method for each semantic unit is deletion.

[0097] For example, in the previous example, based on the sequence S = [S[1]S[2]S[3]] of the reference statement "How to learn programming?", an initial operation sequence O = [O[1]O[2]O[3]] can be created, where O[1] = 0; O[2] = 0; O[3] = 0. Then, according to the order of each semantic unit in the reference statement S, the operation values ​​of the operation methods corresponding to each semantic unit are combined to obtain the reference operation sequence O = [O[1]O[2]O[3]], where O[1] = 2; O[2] = 1; O[3] = 1; that is, O = [21 1].

[0098] Furthermore, in steps 201 to 203 above, a corresponding reference operation mode is generated by determining the operation mode of each semantic unit, and a reference operation sequence is obtained based on the reference operation mode. Then, the reference statement and the corresponding reference operation sequence can be used as feature data.

[0099] Step 103: Use all feature data corresponding to the M reference statements to train a generative pre-training-2 (GPT-2) model to obtain the operation sequence generation model.

[0100] For details, please refer to Figure 5 This is a schematic flowchart illustrating a method for generating an operation sequence generation model according to an embodiment of the present invention. The method includes the following steps:

[0101] Step 501: Perform data processing on the feature data.

[0102] Specifically, the purpose of data processing is to standardize the format of each feature data. This format can be set by those skilled in the art based on experience, or it can be set according to actual needs, without limitation.

[0103] In one possible implementation, the format can be set as: {"query": reference statement S, "labels": reference operation sequence O}. For example, in the previous example, the reference statement S is "How to learn programming?", and its reference operation sequence is O = [21 1]. After processing the feature data, the feature data in a unified format is obtained as: {"query": How to learn programming?, "labels": [2 1 1]}.

[0104] Alternatively, after standardizing the format of each feature data point, each feature data point can be serialized. For example, TensorFlow can be used to serialize each feature data point into a TFRecord format file. TFRecord is the dataset storage format in TensorFlow, and writing and reading it is equivalent to the serialization and deserialization processes. For details regarding TensorFlow and TFRecord, please refer to the existing technology descriptions; this application will not provide specific details therein.

[0105] Step 502: Train the operation sequence generation model.

[0106] Specifically, the GPT-2 model can be trained using all feature data corresponding to M reference sentences. The GPT-2 model uses an open-source Chinese version with 1.5 billion parameters, including a decoder in a 48-layer transformer. Trained on a large amount of text data, the GPT-2 model already possesses a certain level of language understanding and generation capabilities.

[0107] Afterwards, all feature data can be input into the GPT-2 model, enabling the model to learn the correspondence between "query" and "labels" in the feature data, thus allowing the model to learn to output the corresponding operation method based on the input statement.

[0108] Through steps 501 and 502 above, the operation sequence generation model is obtained. In this scheme, the amount of operation sequence data is relatively small, and the corresponding amount of feature data is also relatively small. Therefore, the dataset of similar sentences used to train the operation sequence generation model is also smaller, and this dataset of similar sentences is easy to collect, greatly reducing labor costs. Moreover, using all the feature data with a smaller amount of data can also improve the processing complexity when training the operation sequence generation model, thereby improving the efficiency of model training. In addition, the operation sequence generation model in this scheme is the GPT-2 model, which has stronger language understanding ability, higher generation quality, lower inference cost, and lower computational cost, making this scheme more feasible in practical applications.

[0109] The above content details how to generate an operation sequence generation model. The following section will introduce how to use this model to generate similar statements corresponding to the original statements.

[0110] like Figure 6 The above is a schematic flowchart of a method for generating similar statements provided by an embodiment of the present invention. The method includes the following steps:

[0111] Step 601: Obtain the original statement.

[0112] Step 602: Input the original statement into the operation sequence generation model to obtain the target operation sequence output by the operation sequence generation model.

[0113] The target operation sequence can be one or multiple operation sequences. Specifically, the original statement is input into the operation sequence generation model, and the required number of similar statements, N (where N is a positive integer), is set. The operation sequence generation model then outputs N target operation sequences.

[0114] When the operation sequence generation model is the GPT-2 model, after inputting the original sentence into the GPT-2 model, the GPT-2 model generates only one word (token) at a time, appends this token to the previously generated word, and uses it as the new input for the next step, generating N target operation sequences. By operating the original sentence according to the N target operation sequences, N similar sentences that are most similar to the original sentence can be obtained.

[0115] Step 603: Perform the operation on the original statement according to the operation method indicated by the target operation sequence to obtain the similar statement corresponding to the original statement.

[0116] In detail, after obtaining the target operation sequence, the original statement can be operated in the reverse way of generating the reference operation sequence in step 203 above, according to the operation method indicated by the target operation sequence, to obtain N similar statements corresponding to the original statement.

[0117] In the methods described above, since the input to the operation sequence generation model is text and the output is a sequence, and the amount of data in the operation sequence is relatively small, the operation sequence generation model needs to process less information and can stably and accurately generate the target operation sequence. In contrast, existing technologies typically input the original sentence into a similar sentence generation model to directly obtain similar sentences corresponding to the original sentence. Because the input and output of such similar sentence generation models are both text, they need to process more information, making it difficult to stably and accurately generate similar sentences corresponding to the original sentence. This results in similar sentences that are prone to errors or semantic differences from the original sentence. However, this solution first stably and accurately generates the target operation sequence, and then performs operations on the original sentence according to the target operation sequence, thus stably and accurately generating similar sentences corresponding to the original sentence. The probability of similar sentences generated using this method being incorrect or semantically different from the original sentence is low.

[0118] Based on the same inventive concept described above, the present invention also provides a similar sentence generation device, which can perform the methods described in the embodiments of the invention. The structure of the similar sentence generation device provided by the present invention can be found in [reference needed]. Figure 7The similar statement generation device 700 includes an acquisition unit 701 and a processing unit 702. The acquisition unit 701 acquires the original statement; the processing unit 702 inputs the original statement into an operation sequence generation model to obtain the target operation sequence output by the operation sequence generation model. The operation sequence generation model is trained based on a reference statement, similar statements corresponding to the reference statement, and a reference operation sequence used to convert the reference statement into a similar statement. The reference operation sequence identifies the operation methods used to obtain similar statements corresponding to the reference statement. The original statement is operated according to the operation methods indicated by the target operation sequence to obtain the similar statements corresponding to the original statement.

[0119] For a more detailed description of the acquisition unit 701 and the processing unit 702 described above, please refer to [reference needed]. Figure 6 The relevant descriptions in the method embodiments shown are directly obtained and will not be repeated here.

[0120] Based on the same technical concept, the present invention also provides a computing device, such as... Figure 8 As shown, the computing device 800 includes at least one processor 801 and a memory 802 connected to the at least one processor. The specific connection medium between the processor 801 and the memory 802 is not limited in this invention. Figure 8 Taking the connection between the processor 801 and the memory 802 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0121] In this invention, the memory 802 stores instructions that can be executed by at least one processor 801. By executing the instructions stored in the memory 802, at least one processor 801 can perform the steps included in the aforementioned similar statement generation method.

[0122] The processor 801 is the control center of the computing device. It can connect to various parts of the computing device using various interfaces and lines. By running or executing instructions stored in the memory 802 and calling data stored in the memory 802, it can generate similar statements. Optionally, the processor 801 may include one or more processing units. The processor 801 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles issuing instructions. It is understood that the modem processor may not be integrated into the processor 801. In some embodiments, the processor 801 and the memory 802 may be implemented on the same chip; in some embodiments, they may be implemented on separate chips.

[0123] Processor 801 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the method for generating similar statements can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0124] Memory 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 802 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 802 can be any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but is not limited thereto. Memory 802 in this invention can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0125] Based on the same technical concept, the present invention also provides a computer-readable storage medium storing a computer program executable by a computing device, which, when run on the computing device, causes the computing device to perform the steps of the above-described similar statement generation method.

[0126] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0130] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for generating similar sentences, characterized in that, include: Get the original statement; The original statement is input into the operation sequence generation model to obtain the target operation sequence output by the operation sequence generation model. The operation sequence generation model is trained based on a reference statement, similar statements corresponding to the reference statement, and a reference operation sequence used to convert the reference statement into the similar statement. The reference operation sequence is used to identify the operation method of operating the reference statement to obtain the similar statement corresponding to the reference statement. The original statement is operated according to the operation method indicated by the target operation sequence to obtain a similar statement corresponding to the original statement; The step of operating the original statement according to the operation method indicated by the target operation sequence to obtain a similar statement corresponding to the original statement includes: When the operation mode indicated by the target operation sequence includes insertion, one or more candidate semantic units are selected from the phrase set and inserted into the original statement, wherein the total length of the one or more candidate semantic units is less than a second threshold. The operation sequence generation model is constructed in the following manner: Get M reference statements and the similar statements corresponding to each of the M reference statements, where M is a positive integer, and the number of similar statements corresponding to each reference statement is greater than a threshold. For each of the M reference statements, determine the operation method to convert the reference statement into each similar statement corresponding to the reference statement. Any operation method refers to the operation performed to convert the reference statement into the corresponding similar statement. Generate a corresponding reference operation sequence according to the operation method. Use the reference statement and the corresponding reference operation sequence as feature data. The generative pre-trained Transform-2 model GPT-2 is trained using all feature data corresponding to the M reference statements to obtain the operation sequence generation model.

2. The method for generating similar sentences according to claim 1, characterized in that, The step of determining the operation method for converting the reference statement into each similar statement corresponding to the reference statement, and generating a corresponding reference operation sequence based on the operation method, includes: The reference statement and each similar statement are segmented into words to obtain the semantic units of the reference statement and each semantic unit of each similar statement. For each semantic unit in the reference statement, the semantic unit is compared with each semantic unit in each similar statement to determine the operation method for the semantic unit when converting the reference statement into each similar statement; The reference operation sequence is obtained by combining the operation methods corresponding to each semantic unit according to the order of the semantic units in the reference statement.

3. The method for generating similar sentences according to claim 2, characterized in that, The operation method includes one or more of the following: Delete, keep, insert, swap.

4. The method for generating similar sentences according to claim 2, characterized in that, After performing word segmentation on the reference statement and each similar statement, the process further includes: If it is determined that the reference statement and the similar statement contain the same punctuation mark, and the content of the reference statement before the punctuation mark is the same as the content of the similar statement after the punctuation mark, and the content of the reference statement after the punctuation mark is the same as the content of the similar statement before the punctuation mark, then the operation mode corresponding to the reference statement is determined to be swap.

5. The method for generating similar sentences according to claim 2, characterized in that, The method of determining the operation on the semantic unit when converting the reference statement into each similar statement includes: The semantic units of the reference statement are traversed sequentially. For each semantic unit currently traversed, if it is determined that the semantic unit is the same as a semantic unit in a similar statement, the operation mode of the semantic unit is determined to be "reserved".

6. The method for generating similar sentences according to claim 5, characterized in that, Before comparing each semantic unit in the reference statement with each semantic unit in each similar statement, the method further includes: Obtain the candidate semantic units that are different from the corresponding reference statements in each similar statement; Determine the frequency of each candidate semantic unit among all candidate semantic units, and when the frequency is greater than a first threshold, add the candidate semantic unit to the phrase set; After determining that the operation mode of the semantic unit is to be retained, the method further includes: If there is a semantic unit in the similar statement that does not exist in the reference statement before the semantic unit, then the phrase set is searched based on the semantic unit that does not exist in the reference statement. When the phrase set contains a semantic unit that does not exist in the reference statement, the operation mode for determining the semantic unit that does not exist in the reference statement is insertion. When the phrase set does not contain a semantic unit that does not exist in the reference statement, the operation mode for determining the semantic unit that does not exist is deletion.

7. A similar sentence generation device, characterized in that, include: The retrieval unit is used to retrieve the original statement; The processing unit is configured to input the original statement into an operation sequence generation model to obtain a target operation sequence output by the operation sequence generation model; the operation sequence generation model is trained based on a reference statement, similar statements corresponding to the reference statement, and a reference operation sequence used to convert the reference statement into the similar statement, wherein the reference operation sequence is used to identify the operation method of operating the reference statement to obtain the similar statement corresponding to the reference statement; the original statement is operated according to the operation method indicated by the target operation sequence to obtain the similar statement corresponding to the original statement; The processing unit is specifically configured to select one or more candidate semantic units from the phrase set and insert them into the original statement when the operation mode indicated by the target operation sequence includes insertion, wherein the total length of the one or more candidate semantic units is less than a second threshold. The operation sequence generation model is constructed in the following manner: Find M reference statements and the similar statements corresponding to each of the M reference statements, where M is a positive integer; For each of the M reference statements, determine the operation method to convert the reference statement into each similar statement corresponding to the reference statement, and generate a corresponding reference operation sequence according to the operation method. The reference statement and the corresponding reference operation sequence are used as feature data. The generative pre-trained Transform-2 model GPT-2 is trained using all feature data corresponding to the M reference statements to obtain the operation sequence generation model.

8. A computing device, characterized in that, It includes at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 6.

Images