Text processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a text processing method and device, a storage medium and electronic equipment. The method comprises the following steps: decoding a position lexical element behind each lexical element in a first lexical element sequence to obtain a second lexical element sequence; the M lexical elements in the first lexical element sequence are verified through the second lexical element sequence, the first target lexical element and all the lexical elements before the first target lexical element are verified successfully, and the second target lexical element is located behind the first target lexical element and verification fails; obtaining a third lexical element sequence, wherein the lexical element at the first position in the third lexical element sequence belongs to the second lexical element sequence and is the lexical element at the position behind the first target lexical element; decoding a position lexical element behind each lexical element in the third lexical element sequence to obtain a fourth lexical element sequence; the second seed sequence used for decoding the position lexical element after the third target lexical element in the third lexical element sequence is the same as the first seed sequence used for decoding the position lexical element after the second target lexical element, and the technical problem that the accuracy of text processing is low is solved.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to a text processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] In related technologies, during text prediction processing, a naive threshold-based sampling method is often used to determine the authenticity of lexical units (used to determine the generated words or characters). The lexical units that pass the authenticity determination will be directly used for subsequent text generation.

[0003] However, the joint probability of the text generated by this sampling method is not globally optimal, which affects the accuracy of the subsequently generated text, resulting in low accuracy of text processing.

[0004] Therefore, there is a technical problem with low accuracy in text processing in related technologies. Summary of the Invention

[0005] This application provides a text processing method, apparatus, storage medium, and electronic device to at least address the technical problem of low accuracy in text processing in related technologies.

[0006] According to one aspect of the embodiments of this application, a text processing method is provided, comprising: for input text, obtaining a first word element sequence for obtaining output text, wherein the word element at the first position in the first word element sequence is output by a decoder, and the other M word elements in the first word element sequence are obtained by means other than the decoder, wherein M is a positive integer; decoding the word element at the next position of each word element in the first word element sequence based on M+1 first seed sequences using a decoder to obtain a second word element sequence, wherein a word element in the first word element sequence and a first seed sequence correspond to a word element in the second word element sequence; verifying the M word elements in the first word element sequence using the second word element sequence, wherein, among the M word elements in the first word element sequence, the first target word element and all word elements preceding the first target word element are successfully verified, and the second target word element is located at the first... After a target word fails to be validated, a third word sequence is obtained to obtain the output text. The word at the first position in the third word sequence belongs to the second word sequence and is the word at the position following the first target word. The other M words in the third word sequence are obtained by means other than the decoder. The decoder decodes the word at the position following each word in the third word sequence based on M+1 second seed sequences to obtain a fourth word sequence. One word in the third word sequence and one second seed sequence correspond to one word in the fourth word sequence. The second seed sequence used to decode the word at the position following the third target word in the third word sequence is the same as the first seed sequence used to decode the word at the position following the second target word. The third target word and the second target word correspond to the same position in the output text.

[0007] According to another aspect of the embodiments of this application, a text processing apparatus is also provided, comprising: a first acquisition unit, configured to acquire, for input text, a first word element sequence for obtaining output text, wherein the word element at the first position in the first word element sequence is output by a decoder, and the other M word elements in the first word element sequence are obtained by means other than the decoder, wherein M is a positive integer; a first decoding unit, configured to decode, based on M+1 first seed sequences, the word element at the next position of each word element in the first word element sequence to obtain a second word element sequence, wherein a word element in the first word element sequence and a first seed sequence correspond to obtain a word element in the second word element sequence; and a verification unit, configured to verify the M word elements in the first word element sequence using the second word element sequence, wherein, among the M word elements in the first word element sequence, the first target word element and all word elements preceding the first target word element are successfully verified, and the second target word element is verified. The first target word is located after the first target word and the verification fails; the second acquisition unit is used to acquire the third word sequence for obtaining the output text. The word at the first position in the third word sequence belongs to the second word sequence and is the word at the position after the first target word. The other M words in the third word sequence are obtained by means other than the decoder; the second decoding unit is used to decode the word at the position after each word in the third word sequence based on M+1 second seed sequences to obtain the fourth word sequence. One word in the third word sequence and one second seed sequence correspond to one word in the fourth word sequence; the second seed sequence used to decode the word at the position after the third target word in the third word sequence is the same as the first seed sequence used to decode the word at the position after the second target word. The third target word and the second target word correspond to the same position in the output text.

[0008] According to another aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program / instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program / instructions from the computer-readable storage medium, and executes the computer program / instructions, causing the computer device to perform the text processing method described above.

[0009] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-described text processing method through the computer program.

[0010] In this embodiment, the decoder first obtains the first word in the first word sequence of the output text, and the decoder obtains the other M words after the first word in the first word sequence through other means. In this way, the first word sequence of the output text can be obtained quickly without waiting for the decoder to output word by word.

[0011] Furthermore, the decoder decodes the word at the next position after each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence. The second word sequence is then used to verify the other M words after the first position in the first word sequence to obtain multiple consecutive words that are successfully verified and words that fail to be verified.

[0012] Subsequently, a third word sequence is obtained by combining the word corresponding to the position of the word that failed to be verified in the second word sequence with M word elements obtained through other means besides the decoder. The decoder then decodes the word at the next position of each word in the third word sequence based on M+1 second seed sequences to obtain a fourth word sequence. The fourth word sequence is then used to verify the other M word elements after the first position in the third word sequence.

[0013] In the above process, in decoding the third word sequence, the second seed sequence used for the word at the same position in the output text corresponding to the word that failed to be verified using the second word sequence is the same as the first seed sequence used for the word at the next position after the word that failed to be verified using the second word sequence. This ensures that in the case of verification failure, the random seed sequence of the decoding process for the word that failed to be verified is preserved, so that it has the sampling distribution property corresponding to the expected output word, thereby ensuring that the joint probability of the generated text is globally optimal, thus achieving the technical effect of improving the accuracy of text processing. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0015] Figure 1 This is a schematic diagram of an application environment for an optional text processing method according to an embodiment of this application;

[0016] Figure 2 This is a schematic diagram of the flow of an optional text processing method according to an embodiment of this application;

[0017] Figure 3 This is a schematic diagram of an optional text processing method according to an embodiment of this application;

[0018] Figure 4 This is a schematic diagram of an optional text processing method according to an embodiment of this application;

[0019] Figure 5 This is a schematic diagram of an optional text processing method according to an embodiment of this application;

[0020] Figure 6 This is a schematic diagram of an optional text processing method according to an embodiment of this application;

[0021] Figure 7 This is a schematic diagram of an optional text processing method according to an embodiment of this application;

[0022] Figure 8 This is a schematic diagram of an optional text processing method according to an embodiment of this application;

[0023] Figure 9 This is a schematic diagram of an optional text processing method according to an embodiment of this application;

[0024] Figure 10 This is a schematic diagram of an optional text processing apparatus according to an embodiment of this application;

[0025] Figure 11 A schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] According to one aspect of the embodiments of this application, a text processing method is provided. Optionally, as an optional implementation, the above-described text processing method may be applied to, but is not limited to, [examples of text processing methods]. Figure 1 The environment shown may include, but is not limited to, a client 102 and a server 112. The client 102 may include, but is not limited to, a display 104, a processor 106, and a memory 108. The server 112 includes a database 114 and a processing engine 116.

[0029] The specific process can be summarized in the following steps:

[0030] In step S102, the client 102 obtains a text processing request, wherein the text processing request is used to request the processing of the input text to obtain the output text;

[0031] In steps S104-S106, client 102 sends a text processing request to server 112;

[0032] In step S108, for the input text, server 112 obtains the first word sequence for obtaining the output text. The word at the first position in the first word sequence is output by the decoder, and the other M words in the first word sequence are obtained by means other than the decoder, where M is a positive integer.

[0033] In step S110, server 112 decodes the word at the next position of each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence. One word in the first word sequence and one first seed sequence correspond to one word in the second word sequence.

[0034] In step S112, server 112 verifies M words in the first word sequence using the second word sequence. Among the M words in the first word sequence, the first target word and all words before the first target word are verified successfully, while the second target word is located after the first target word and fails to be verified.

[0035] In step S114, server 112 obtains the third word sequence for obtaining the output text. The word at the first position in the third word sequence belongs to the second word sequence and is the word at the position following the first target word. The other M words in the third word sequence are obtained by means other than the decoder.

[0036] In step S116, server 112 decodes the word at the position following each word in the third word sequence based on M+1 second seed sequences using a decoder to obtain a fourth word sequence. A word in the third word sequence and a second seed sequence correspond to obtain a word in the fourth word sequence. The words in the second word sequence and the words in the fourth word sequence are used to verify the target word. The target word is converted into the output text after format conversion.

[0037] The second seed sequence used to decode the word at the position following the third target word in the third word sequence is the same as the first seed sequence used to decode the word at the position following the second target word. The third target word and the second target word correspond to the same position in the output text.

[0038] In steps S118-S120, the output text is sent to the client 102 via the network 110. The processor 106 in the client 102 is used to receive and process the output text, display the results on the display 104, and store the results in the memory 108.

[0039] remove Figure 1 Beyond the examples shown, the above steps can be completed independently by the client or server, or collaboratively by both, such as by client 102 executing steps S108 to S116, thereby reducing the processing load on server 112. Client 102 includes, but is not limited to, laptops, tablets, desktop computers, smart TVs, etc., and this application does not limit the specific implementation of client 102. Server 112 can be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0040] Alternatively, as an alternative implementation method, such as Figure 2 As shown, text processing methods can be executed by electronic devices, such as... Figure 1 The client or server shown includes the following specific steps:

[0041] Step S202: For the input text, obtain the first word sequence for obtaining the output text. The word at the first position in the first word sequence is output by the decoder, and the other M words in the first word sequence are obtained by means other than the decoder, where M is a positive integer.

[0042] Step S204: The decoder decodes the word at the next position of each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence. One word in the first word sequence and one first seed sequence correspond to one word in the second word sequence.

[0043] Step S206: Verify the M words in the first word sequence using the second word sequence. Among the M words in the first word sequence, the first target word and all words before the first target word are successfully verified, while the second target word is located after the first target word and fails to be verified.

[0044] Step S208: Obtain the third word sequence for obtaining the output text. The word at the first position in the third word sequence belongs to the second word sequence and is the word at the next position after the first target word. The other M words in the third word sequence are obtained by means other than the decoder.

[0045] Step S210: The decoder decodes the word at the position following each word in the third word sequence based on M+1 second seed sequences to obtain the fourth word sequence. One word in the third word sequence and one second seed sequence correspond to one word in the fourth word sequence. The second seed sequence used to decode the word at the position following the third target word in the third word sequence is the same as the first seed sequence used to decode the word at the position following the second target word. The third target word and the second target word correspond to the same position in the output text.

[0046] Optionally, in this embodiment, the above text processing method can be applied, but is not limited to, in a text prediction scenario based on parallel decoding. In this scenario, the user initiates a text processing request to the prediction model. The text processing request carries the user's input text, which can be, but is not limited to, consultation or question text input by the user. The text processing request is used to request the input text to be processed to obtain output text, which can be, but is not limited to, the answer text to the corresponding information or question text.

[0047] To illustrate further, if the input text is "What are some fun places in area A?", the corresponding output text after processing could be "Fun places in area A include attractions a, b, c, etc."

[0048] Optionally, in this embodiment, the prediction model may be, but is not limited to, a model that includes a decoder and other decoding modules besides the decoder. The decoder may be, but is not limited to, used to obtain the first word in the first word sequence of the output text, and the other decoding modules may be, but are not limited to, used to output the other M words after the first word in the first word sequence, where M is a positive integer.

[0049] Optionally, in this embodiment, the process of the decoder outputting tokens can be, but is not limited to, based on autoregressive decoding. Further examples are provided, such as... Figure 3As shown, the input text 302 "What is the weather suitable for today?" is input to the decoder 304. After receiving the input text 302, the decoder 304 will make a prediction of each character (for ease of intuitive understanding, it is assumed in this embodiment that each character is a single word / token, which will not be elaborated on later), and then output the text 306 "Today is suitable for cycling".

[0050] Optionally, in this embodiment, other decoding modules may be, but are not limited to, decoding heads for parallel decoding, such as Medusa heads. It is understood that the Medusa head is an independent module; based on the decoder, N Medusa heads can be integrated to obtain a prediction model. The decoder predicts the word at the first position in the first word sequence of the output text, and the N Medusa heads predict the word at the second position, the third position, ..., the Nth candidate output word in the first word sequence, respectively.

[0051] To further illustrate, a schematic diagram of the decoding prediction of an optional prediction model integrating two Medusa heads is shown below. Figure 4 As shown, the input text 402 "What is the weather suitable for doing today?" is input into the prediction model 404. After receiving the input text 402, the prediction model 404 will make a prediction of 3 characters at once. The decoder predicts the word unit at the first position, Medusa head 1 predicts the word unit at the second position, and Medusa head 2 predicts the word unit at the third position, thus determining the first word unit sequence as "Today is suitable". And so on, the third word unit sequence "Go cycling" is obtained. If both the first word unit sequence and the third word unit sequence are successfully verified, the output text 406 "Today is suitable for cycling" is obtained.

[0052] Optionally, in this embodiment, the decoder decodes the word at the next position of each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence. This can include, but is not limited to, decoding the word at the first i positions in the first word sequence and the i-th first seed sequence in the M+1 first seed sequences to obtain the (i+1)-th second word in the second word sequence, where i is a positive integer (for ease of understanding, the word included in the first word sequence will be referred to as the first word, and the word included in the second word sequence will be referred to as the second word).

[0053] Optionally, in this embodiment, verifying the M tokens in the first token sequence using the second token sequence may include, but is not limited to, using the i-th second token among the first M second tokens in the second token sequence to verify the i-th first token among the M first tokens in the first token sequence.

[0054] Optionally, in this embodiment, the i-th second word element among the first M second word elements in the second word element sequence is used to verify the i-th first word element among the M first word elements in the first word element sequence. This may include, but is not limited to, determining that the i-th first word element is successfully verified when the i-th second word element is the same as the i-th first word element, and determining that the i-th first word element fails to be verified when the i-th second word element is different from the i-th first word element.

[0055] Optionally, in this embodiment, if all words before the first target word and the first target word are successfully verified, and the second target word is located after the first target word and fails to be verified (e.g., the second target word is the next word after the first target word and fails to be verified), the third word sequence of the input text is obtained. The word at the first position in the third word sequence is the word that belongs to the second word sequence and is the word at the next position after the first target word, that is, the second word corresponding to the position of the word that fails to be verified for the first time among the M words. The other M words after the first position in the third word sequence are obtained by means other than the decoder.

[0056] Optionally, in this embodiment, the decoder decodes the word at the next position after each word in the third word sequence based on M+1 second seed sequences to obtain the fourth word sequence. This can include, but is not limited to, decoding the word at the first i positions in the third word sequence and the i-th second seed sequence in the M+1 second seed sequences to obtain the (i+1)-th fourth word in the fourth word sequence, where i is a positive integer (for ease of understanding, the word included in the third word sequence will be referred to as the third word, and the word included in the fourth word sequence will be referred to as the fourth word).

[0057] It should be noted that the second seed sequence used to decode the word at the position following the third target word in the third word sequence is the same as the first seed sequence used to decode the word at the position following the second target word. The third target word and the second target word correspond to the same position in the output text.

[0058] To further illustrate, such as Figure 5 As shown, the first word sequence "A, B, C, D" of the input text is obtained. The word A at the first position in the first word sequence is output by the decoder, and the other three words B, C, and D are obtained by other methods outside the decoder (such as parallel decoding).

[0059] The decoder decodes the word at the next position after each word in the first word sequence based on four first seed sequences "a, b, c, d", to obtain the second word sequence "B~, C~, D~". The second word B~ is obtained by decoding the previous word and the first first seed sequence a in the first word sequence and is used to verify the first word B in the first word sequence. The word C~ is obtained by decoding the first two words and the second first seed sequence b in the first word sequence and is used to verify the first word C in the first word sequence. The word D~ is obtained by decoding the first three words and the third first seed sequence c in the first word sequence and is used to verify the first word D in the first word sequence. The word E~ is obtained by decoding the first four words and the fourth first seed sequence d in the first word sequence and is used to verify the word at the next position after the first word D in the output text.

[0060] Furthermore, in Figure 5 Based on this, assuming the verification result indicates that the second word element B is the same as the first word element B (i.e., the first word element B was successfully verified), the second word element C is the same as the first word element C (i.e., the first word element C was successfully verified), and the second word element D is different from the first word element D (i.e., the first word element D failed to be verified), it can be determined that among the words in the first word element sequence, the first word element C and all words preceding it were successfully verified, while the first word element D, which is located after the first word element C, failed to be verified.

[0061] Furthermore, such as Figure 6 As shown, the third word sequence of the input text is obtained. The number of words included in the third word sequence is the same as that in the first word sequence. The third word at the first position of the third word sequence is the word that belongs to the second word sequence and is the word one position after the first word C. That is, the third word at the first position of the third word sequence is word D~. The third words at other positions of the third word sequence are "E, F, G".

[0062] like Figure 6 As shown, M+1 second seed sequences are obtained. The first seed sequence among the M+1 second seed sequences is the first seed sequence d used to decode and generate the second word E. The other M second seed sequences are the newly generated seed sequences "e, f, g".

[0063] It is understandable that, during the decoding of the third word sequence, the second seed sequence d used for the third word E at the next position after the third target word D is the same seed sequence d used for decoding the first word E at the next position after the second target word D in the first word sequence.

[0064] It should also be noted that the fourth word sequence is used to verify the words after the first word in the third word sequence, and the word sequence and seed sequence of the output text in the next round are obtained based on the verification results.

[0065] The embodiments provided in this application first obtain the first word element in the first word element sequence of the output text using a decoder, and then obtain the remaining M words after the first word element in the first word element sequence using other methods outside the decoder. This allows for rapid acquisition of the first word element sequence of the output text without waiting for the decoder to output words one by one. Further, the decoder decodes the word element at the next position after each word element in the first word element sequence based on M+1 first seed sequences to obtain a second word element sequence. The remaining M words after the first word element in the first word element sequence are then verified using the second word element sequence to obtain multiple consecutive successfully verified words and words that fail verification. Subsequently, a third word element sequence is obtained by combining the word element corresponding to the position of the word element that failed verification in the second word element sequence with the M words obtained through other methods outside the decoder. The decoder then decodes the word element at the next position after each word element in the third word element sequence based on M+1 second seed sequences to obtain a fourth word element sequence. The remaining M words after the first word element in the third word element sequence are then verified using the fourth word element sequence.

[0066] In the above process, in decoding the third word sequence, the second seed sequence used for the word at the same position in the output text corresponding to the word that failed to be verified using the second word sequence is the same as the first seed sequence used for the word at the next position after the word that failed to be verified using the second word sequence. This ensures that in the case of verification failure, the random seed sequence of the decoding process for the word that failed to be verified is preserved, so that it has the sampling distribution property corresponding to the expected output word, thereby ensuring that the joint probability of the generated text is globally optimal, thus achieving the technical effect of improving the accuracy of text processing.

[0067] As an optional approach, the M tokens in the first token sequence are validated using the second token sequence, including:

[0068] The Nth second word in the first M second word sequence is used sequentially to verify the Nth first word in the first M first word sequence, where N is a positive integer;

[0069] If a reference word that is different from the corresponding second word appears for the first time among the M first word words, the reference word word verification is determined to be a failure, the reference word word word is determined to be the second target word word word, and the word word word one position before the reference word word word word among the M first word words word ...

[0070] Optionally, in this embodiment, the i-th second word element among the first M second word elements in the second word element sequence is used to verify the i-th first word element among the M first word elements in the first word element sequence.

[0071] Optionally, in this embodiment, if the i-th second word is the same as the i-th first word, the i-th first word is determined to be successfully verified; if the i-th second word is different from the i-th first word, the i-th first word is determined to be unverified.

[0072] It should be noted that the (M+1)th second word in the second word sequence is used to verify the word at the next word position of the last word position in the first word sequence of the output text.

[0073] Optionally, in this embodiment, if a reference word that is different from the corresponding second word appears for the first time among the M first word words, it is determined that the reference word word verification failed, and it is determined that all word words before the reference word word in the M first word words have been successfully verified. The word word at the position before the reference word word word is determined as the first target word word word, and the reference word word word is determined as the second target word word word word.

[0074] As an optional approach, obtaining the third word sequence used to obtain the output text includes:

[0075] The second word corresponding to the reference word and the other M words obtained by means other than the decoder are determined as the third word sequence, wherein the second word corresponding to the reference word is the word at the first position in the third word sequence;

[0076] Before decoding the word at the position following each word in the third word sequence based on M+1 second seed sequences using a decoder to obtain the fourth word sequence, the method further includes:

[0077] Obtain M+1 second seed sequences, where the reference seed sequence is the first second seed sequence among the M+1 second seed sequences, and the reference seed sequence is the first seed sequence among the M+1 first seed sequences used to generate reference lexical units.

[0078] Optionally, in this embodiment, in the first lexical sequence, all lexical units preceding the reference lexical unit are successfully verified, while the reference lexical unit fails verification. The second lexical unit corresponding to the reference lexical unit in the second lexical sequence is determined as the lexical unit at the first position in the third lexical sequence, wherein the second lexical unit corresponding to the reference lexical unit is the lexical unit in the second lexical sequence used to verify the reference lexical unit.

[0079] It should be noted that the third word sequence may include, but is not limited to, the same number of words as the first word sequence. The other M words after the first position in the third word sequence are obtained by means other than the decoder, such as parallel decoding.

[0080] Optionally, in this embodiment, before decoding the word at the next position of each word in the third word sequence based on M+1 second seed sequences to obtain the fourth word sequence, M+1 second seed sequences are obtained, wherein the first second seed sequence among the M+1 second seed sequences is the first seed sequence among the M+1 first seed sequences used to generate reference words.

[0081] Understandably, if the reference word verification fails in the previous round, the second word corresponding to the reference word will be determined as the first word of the third word sequence in the next round of verification (no verification required). The offset of the first seed sequence used to generate the reference word will be retained at the beginning of the third word sequence before the next round of verification. The fourth word corresponding to the third word at the second position of the third word sequence will be used to verify the third word at the second position.

[0082] The embodiments provided in this application ensure that, in the event of a verification failure, the random seed sequence of the decoding process for the failed tokens is preserved, so that it has the sampling distribution attribute corresponding to the expected output tokens, thereby ensuring that the joint probability of the generated text is globally optimal, thus achieving the technical effect of improving the accuracy of text processing.

[0083] As an optional approach, after decoding the word at the position following each word in the first word sequence based on M+1 first seed sequences using a decoder to obtain the second word sequence, the method further includes:

[0084] The M words in the first word sequence are verified by the second word sequence, and all words in the M words of the first word sequence are successfully verified.

[0085] Obtain the fifth word sequence used to obtain the output text. The word at the first position in the fifth word sequence is the last word in the M+1 words of the second word sequence. The other M words in the fifth word sequence are obtained by means other than the decoder.

[0086] The decoder decodes the word in the fifth word sequence one position after each word in the fifth word sequence based on M+1 third seed sequences to obtain the sixth word sequence. A word in the fifth word sequence and a third seed sequence correspond to a word in the sixth word sequence.

[0087] Optionally, in this embodiment, in the M words of the first word sequence, all words are successfully verified, which may include, but is not limited to, the i-th second word in the first M words of the second word sequence being the same as the i-th first word in the M words of the first word sequence.

[0088] Optionally, in this embodiment, the last word (the (M+1)th word) in the M+1 words of the second word sequence is determined as the word at the first position of the fifth word sequence, wherein the other M words corresponding to the M positions after the first position of the fifth word sequence are obtained by means other than the decoder.

[0089] It should be noted that the sixth word sequence is used to verify the words after the first word in the fifth word sequence, and the word sequence and seed sequence of the output text in the next round are obtained based on the verification results.

[0090] To illustrate further, in Figure 5 Based on this, assuming the verification results indicate that the second word element B is the same as the first word element B (i.e., the first word element B was successfully verified), the second word element C is the same as the first word element C (i.e., the first word element C was successfully verified), and the second word element D is the same as the first word element D (i.e., the first word element D was successfully verified), it can be determined that all words in the first word element sequence were successfully verified.

[0091] Furthermore, such as Figure 7 As shown, the fifth word sequence of the input text is obtained. The number of words included in the fifth word sequence is the same as that in the first word sequence. The word at the first position of the fifth word sequence belongs to the second word sequence and is the last word. That is, the word at the first position of the fifth word sequence is word E~. The words at other positions of the third word sequence are "F, G, H, I".

[0092] like Figure 7 As shown, obtain M+1 third seed sequences "e, f, g, h", where the third seed sequence e is used to decode and generate the sixth word F (used to verify the fifth word F), the third seed sequence f is used to decode and generate the sixth word G (used to verify the fifth word G), the third seed sequence g is used to decode and generate the sixth word H (used to verify the fifth word H), and the third seed sequence h is used to decode and generate the sixth word I (used to verify the fifth word I).

[0093] As an optional approach, a decoder decodes the word at the position following each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence, including:

[0094] During the decoding process of the word at the position following the Qth word in the first word sequence, regression prediction is performed based on the first Q words in the first word sequence to obtain P output words. Each output word corresponds to a probability score, which is used to represent the probability that the output word is the word at the position following the Qth word. Q and P are positive integers.

[0095] The P probability scores corresponding to the P output words are integrated with the Kth first seed sequence in the M+1 first seed sequences to obtain the P target probability scores.

[0096] The output word with the highest target probability score among the P target probability scores is determined as the second word corresponding to the next position of each word in the first word sequence. The second word sequence includes the second word.

[0097] Optionally, in this embodiment, the first Q words in the first word sequence are regressed and predicted by the decoder to obtain P output words. Each output word corresponds to a probability score, which represents the probability of the standard output word at the corresponding output position. Based on the P probability scores corresponding to the P output words, the second word at the position following the Qth word is determined from the P output words.

[0098] Optionally, in this embodiment, the P probability scores corresponding to the P output words are integrated and calculated with the first seed sequence corresponding to the next position of the Qth word to obtain P probability scores. This may include, but is not limited to, dividing each probability score by the random number corresponding to the ordered P random numbers included in the first seed sequence corresponding to the next position of the Qth word to obtain P target probability values.

[0099] To further illustrate, given P probability scores (ordered and arranged from largest to smallest) of 0.5, 0.4, and 0.1, and the first seed sequence at the corresponding position being 1, 2, and 1, the P target probability scores obtained after integration calculation are 0.5, 0.2, and 0.1. Here, 0.5 is obtained by dividing probability score 0.5 by 1, 0.2 is obtained by dividing probability score 0.4 by 2, and 0.1 is obtained by dividing probability score 0.1 by 1.

[0100] It should be noted that the output word with the highest target probability score among the P target probability scores is determined as the second word corresponding to the next position of each word in the first word sequence, and the second word sequence includes the second word.

[0101] As an optional approach, regression prediction is performed based on the first Q words in the first word sequence to obtain P output words, including:

[0102] Regression prediction is performed on the first Q words in the first word sequence to obtain multiple initial output words;

[0103] From multiple initial output words, determine the K output words with the highest probability scores, where K is a positive integer;

[0104] From the K output words, determine the P output words whose sum of probability scores is greater than or equal to a preset threshold.

[0105] Optionally, in this embodiment, the K most likely output words are selected from a plurality of initial output words, and then the P output words whose sum of probability scores is greater than or equal to a preset threshold are selected from the K output words.

[0106] It should be noted that, based on the probability score, the second word corresponding to that position is determined from P output words.

[0107] It should be noted that after determining the sum of the probability scores of P output words that are greater than or equal to the preset threshold, the probability scores of the P output words can be normalized and updated, but not limited to, so as to adjust the data to a uniform range or scale, such as 0 to 1 or -1 to 1, making the data processing more fair and effective.

[0108] To further illustrate, consider P output words, including the first output word, the second output word, and the third output word. The probability score of the first output word is 0.3, the probability score of the second output word is 0.1, and the probability score of the third output word is 0.1. Then, after normalization, the probability score of the first output word can be, but is not limited to, 0.3 / (0.3+0.1+0.1) = 0.6, and the probability scores of the second and third output words after normalization can be, but are not limited to, 0.1 / (0.3+0.1+0.1) = 0.2.

[0109] It should be noted that other normalization methods can also be used, but are not limited to, to scale the probability fractions, such as logarithmic normalization and decimal scaling normalization. This embodiment does not limit this.

[0110] The embodiments provided in this application first determine the top K output words with the highest probability scores, and then determine the P output words whose cumulative probability scores reach a preset threshold. This allows focusing only on output words with higher prediction values, reducing processing time and resource consumption. Furthermore, it ensures that the sampling distribution is the same as that of the autoregressive decoding model, making the probability of the second word obtained based on the P output words whose cumulative probability reaches the preset threshold better, thereby improving the accuracy of the second word sequence. This achieves the technical effect of improving the accuracy of input text processing.

[0111] As an optional approach, for the input text, obtain the first word sequence used to obtain the output text, including:

[0112] The input text is segmented into words to obtain multiple keywords;

[0113] Perform first-sequence format conversion on multiple keywords to obtain multiple word tensors after conversion;

[0114] Perform a first decoding process on multiple word tensors to obtain the word at the first position in the first word sequence, and perform a second decoding process on multiple word tensors to obtain the other M words in the first word sequence;

[0115] After decoding the word at the position following each word in the third word sequence based on M+1 second seed sequences using a decoder to obtain the fourth word sequence, the method further includes:

[0116] The first target word in the second word sequence and the second target word in the fourth word sequence are subjected to second sequence format conversion to obtain the converted natural language text. The first target word is the second word corresponding to the word that was successfully verified in the first word sequence, and the second target word is the fourth word corresponding to the word that was successfully verified in the third word sequence.

[0117] The natural language text is identified as the desired output text.

[0118] Optionally, in this embodiment, the input text (prompt) to be processed is tokenized to obtain multiple keywords, wherein the multiple keywords are converted by a first sequence format to obtain multiple converted word tensors.

[0119] Optionally, in this embodiment, performing a first decoding process on multiple lexical tensors to obtain the lexical at the first position in the first lexical sequence can be used, but is not limited to, to indicate that multiple lexical tensors are input into a neural network decoder to generate an extended first-position output lexical.

[0120] Optionally, in this embodiment, a second decoding process is performed on multiple lexical tensors to obtain candidate output lexical units, which may, but are not limited to, be used to indicate the M lexical units located after the first position in the input text generated by parallel decoding (such as integrating multiple Medusa heads).

[0121] Optionally, in this embodiment, the first word in the first word sequence, the successfully verified word in the first word sequence, and the successfully verified word in the third word sequence are subjected to a second sequence format conversion (detokenization, the process of converting the generated word sequence back to the original text or sentence) to obtain the converted natural language text, and the natural language text is determined as the expected output text.

[0122] As an optional approach, the aforementioned text processing methods can be applied to the decoder scenario to improve sampling consistency and accuracy during parallel decoding. A schematic diagram of the decoder is shown below. Figure 8 As shown, the decoder model includes an embedding layer, a positional encoding layer, a decoder block, a linear layer, an activation function (softmax), and a decoding strategy.

[0123] Optionally, in this embodiment, the embedding layer is used to embed discrete data (such as...) Figure 8 The code transforms 3923, 1933, etc. into low-dimensional continuous vectors; the positional encoding layer provides the model with information about the position of words in the sequence; the decoder block generates the output sequence; the linear layer performs linear transformations, feature transformations, etc.; the activation function transforms a vector or a set of real numbers into a probability distribution such that the value of each element is between 0 and 1, and the sum of all elements is 1, resulting in predicted values ​​(logits); the decoding strategy indicates the method for generating the output sequence.

[0124] Optionally, in this embodiment, the decoding measurement may include, but is not limited to, greedy decoding, selecting the token with the highest logit. Before selecting the final token, some transformations may be applied to the logits. The word with the highest probability is selected as its next word. The advantage is speed, but the disadvantage is that it misses high-probability words hidden behind low-probability words and easily generates repetitive phrases.

[0125] Optionally, in this embodiment, decoding measurement may include, but is not limited to, sampling decoding. The model's logits are treated as a multinomial distribution, and sampling is performed using this distribution. A token is selected from the vocabulary through sampling. The distribution from which sampling is performed can be adjusted first through some simple transformation operations (such as temperature scaling, top-k, and top-p). The next word is randomly selected with conditional probability. When using sampling, text generation itself is no longer deterministic; tokens with low probability also have a chance to be generated. In Top-K sampling, the K words with the highest probabilities are selected, and then the probabilities of these K words are renormalized. Finally, sampling is performed from these K words with renormalized probabilities. In Top-p, sampling is not only performed on the K most likely words, but also on the smallest set of words whose cumulative probability exceeds probability p. Top-p and top-K sampling can be used in combination to produce higher accuracy and higher generation quality than Greedy Search in open-ended language generation.

[0126] Optionally, a decoding diagram of a text processing method based on the above prediction model is shown below. Figure 9 As shown, the decoder may, but is not limited to, using Figure 4 The model structure of prediction model 404 shown illustrates a process where the decoder simultaneously predicts three tokens: "in", "the", and "bus". During the verification process, a specific sampling strategy is applied to extract and verify these three tokens. Ultimately, "in" and "the" are accepted, while "bus" is rejected. Then, the next round of prediction is performed... subsequently predicting tokens such as "car", "this", and "week".

[0127] It's important to note that the decoder itself does not output tokens, but rather logits (the number of logits is the same as the size of the vocabulary; each logit is a numerical vector with dimensions equal to the size of the vocabulary, representing the probability score of each token). The preprocessed logits are converted into a probability distribution using the softmax function, and the next token is sampled from it using the multinomial method.

[0128] To address the randomness issue introduced by the multinomial function in sampling logits, different random seed sequences are maintained based on the verify results to ensure complete consistency with the sampling results and generated answers from the original autoregressive decoding.

[0129] Taking the prediction of two additional tokens as an example: after verification, the best result is that both tokens are accepted, i.e., accept_len = 2. First, three batches of random seed sequences are generated: q_base, q_medusa_head0, and q_medusa_head1.

[0130] Each subsequent decoding step maintains the random seed state and offset based on accept_len.

[0131] Case 1: accept_len = 0 (both additional token verifications fail), q_base = q_medusa_head0, q_medusa_head0 = q_medusa_head1, and a new batch of random seeds is generated for q_medusa_head1.

[0132] Case 2: accept_len = 1 (two additional tokens are predicted and one passes verification), q_base = q_medusa_head1, and a new batch of random seeds is generated for q_medusa_head0 and q_medusa_head1 respectively.

[0133] Case 3: accept_len = 2 (two additional tokens are predicted and both pass the verification), and a new batch of random seeds is generated for q_base, q_medusa_head0, and q_medusa_head1 in sequence.

[0134] After obtaining the corresponding seeds, divide logits by seeds and take the maximum value argmax to obtain the predicted token.

[0135] Optionally, a parallel decoding process based on the above text processing method is as follows:

[0136] Step S1: Load the model weights onto the GPU;

[0137] Step S2: Tokenize the prompt on the CPU and transfer the token tensor to the GPU;

[0138] Step S3: Input the prompt after word segmentation into the neural network decoder to generate the first expanded token;

[0139] Step S4: Different parallel decoding schemes generate candidate tokens according to their own implementation mechanisms;

[0140] Step S5: The first token and candidate tokens are concatenated and appended to the input token sequence, which is then input into the neural network decoder.

[0141] Step S6: Apply a certain decoding strategy to sample, score, and verify the logits output by the decoder to determine the final predicted tokens, the final predicted output token of this round, and the input token of the previous round, and update the decoder input.

[0142] Step S7: Repeat steps S4, S5, and S6 until a stop sequence (e.g., a single end-of-sequence (EOS) token) is generated or the configured maximum sequence length is reached.

[0143] Step S8: The completed tokens are retrieved from the GPU to the CPU and detokenized (detokenization refers to the process of converting the token sequence generated by the model back into the original text or sentence. This may include removing spaces between tokens, adding punctuation marks, restoring abbreviations, etc., to restore the natural language form of the generated text) to obtain the generated text.

[0144] It should be noted that this embodiment introduces a sampling strategy combining top-k and top-p in the token verification process to ensure the same sampling distribution as the top-p in the original autoregressive decoding. To address the randomness introduced by logits sampling, different random seed sequences are maintained based on the verify results to ensure complete consistency between the sampling results and the generated responses in the original autoregressive decoding.

[0145] It is understood that this embodiment optimizes the step of "extracting and verifying candidate tokens generated in this round to decide whether to accept these tokens as real output" in the relevant parallel decoding scheme. By optimizing the sampling strategy for extracting and verifying candidate tokens, the sampling distribution is the same as that in the original autoregressive decoding, thereby improving the accuracy of parallel decoding.

[0146] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0147] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0148] According to another aspect of the embodiments of this application, a text processing apparatus for implementing the above-described text processing method is also provided. For example... Figure 10 As shown, the device includes:

[0149] The first acquisition unit 1002 is used to acquire, for input text, a first word sequence for obtaining output text. The word at the first position in the first word sequence is output by the decoder, and the other M words in the first word sequence are obtained by means other than the decoder, where M is a positive integer.

[0150] The first decoding unit 1004 is used to decode the word at the next position of each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence. One word in the first word sequence and one first seed sequence correspond to one word in the second word sequence.

[0151] The verification unit 1006 is used to verify M words in the first word sequence through the second word sequence, wherein the first target word and all words before the first target word are successfully verified, and the second target word is located after the first target word and fails to be verified.

[0152] The second acquisition unit 1008 is used to acquire a third word sequence for obtaining the output text. The word at the first position in the third word sequence belongs to the second word sequence and is the word at the next position after the first target word. The other M words in the third word sequence are obtained by means other than the decoder.

[0153] The second decoding unit 1010 is used to decode the word at the next position of each word in the third word sequence based on M+1 second seed sequences to obtain the fourth word sequence. One word in the third word sequence and one second seed sequence correspond to one word in the fourth word sequence.

[0154] The second seed sequence used to decode the word at the position following the third target word in the third word sequence is the same as the first seed sequence used to decode the word at the position following the second target word. The third target word and the second target word correspond to the same position in the output text.

[0155] As an optional solution, the above-mentioned verification unit 1006 includes:

[0156] The first verification module is used to verify the Nth first word element in the M first word elements in the first word element sequence by using the Nth second word element in the first word element sequence in the first word element sequence in turn, where N is a positive integer;

[0157] The first determining module is used to determine the reference word verification failure and determine the reference word as the second target word when a reference word that is different from the corresponding second word appears for the first time among the M first word words, and to determine the word word one position before the reference word word among the M first word words as the first target word word.

[0158] As an optional solution, the second acquisition unit 1008 mentioned above includes:

[0159] The second determining module is used to determine the second word corresponding to the reference word and the other M words obtained by means other than the decoder as the third word sequence, wherein the second word corresponding to the reference word is the word at the first position in the third word sequence;

[0160] The device also includes:

[0161] The first acquisition module is used to acquire M+1 second seed sequences before decoding the word at the next position of each word in the third word sequence based on M+1 second seed sequences to obtain the fourth word sequence. The reference seed sequence is the first second seed sequence among the M+1 second seed sequences, and the reference seed sequence is the first seed sequence used to generate the reference word among the M+1 first seed sequences.

[0162] As an optional solution, the above-mentioned device further includes:

[0163] The second verification module is used to decode the word at the next position of each word in the first word sequence based on M+1 first seed sequences using a decoder to obtain the second word sequence, and then verify the M words in the first word sequence using the second word sequence, wherein all words in the M words of the first word sequence are successfully verified.

[0164] The second acquisition module is used to acquire the fifth word sequence for obtaining the output text after decoding the word at the next position of each word in the first word sequence based on M+1 first seed sequences using a decoder. The word at the first position in the fifth word sequence is the last word in the M+1 words of the second word sequence. The other M words in the fifth word sequence are obtained by means other than the decoder.

[0165] The first decoding module is used to decode the word at the next position of each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence, and then decode the word at the next position of each word in the fifth word sequence based on M+1 third seed sequences to obtain the sixth word sequence. One word in the fifth word sequence and one third seed sequence correspond to one word in the sixth word sequence.

[0166] As an optional solution, the first decoding unit 1004 includes:

[0167] The prediction module is used to perform regression prediction based on the first Q words in the first word sequence during the decoding process of the word at the position following the Qth word in the first word sequence by the decoder, and obtain P output words. Each output word corresponds to a probability score, which is used to represent the probability that the output word is the word at the position following the Qth word. Q and P are positive integers.

[0168] The integration module is used to integrate and calculate the P probability scores corresponding to the P output words and the first seed sequence corresponding to the next position of the Qth word to obtain P target probability scores.

[0169] The third determining module is used to determine the output word with the largest target probability score among P target probability scores as the second word corresponding to the next position of each word in the first word sequence, and the second word sequence includes the second word.

[0170] As an optional solution, the above prediction module includes:

[0171] The prediction submodule is used to perform regression prediction on the first Q words in the first word sequence to obtain multiple initial output words;

[0172] The first determining submodule is used to determine the K output words with the highest probability scores from a plurality of initial output words, where K is a positive integer;

[0173] The second determination submodule is used to determine P output words from K output words whose sum of probability scores is greater than or equal to a preset threshold.

[0174] As an optional solution, the first acquisition unit 1002 mentioned above includes:

[0175] The word segmentation module is used to segment the input text into words to obtain multiple keywords;

[0176] The first conversion module is used to perform first sequence format conversion on multiple keywords to obtain multiple word tensors after conversion;

[0177] The second decoding module is used to perform a first decoding process on multiple word tensors to obtain the word at the first position in the first word sequence, and to perform a second decoding process on multiple word tensors to obtain the other M words in the first word sequence.

[0178] The above-mentioned device also includes:

[0179] The second conversion module is used to decode the word at the next position of each word in the third word sequence based on M+1 second seed sequences using a decoder to obtain the fourth word sequence. Then, it performs a second sequence format conversion on the word at the first position in the first word sequence, the first target word in the first word sequence, and the second target word in the third word sequence to obtain the converted natural language text. The first target word is the word that has been successfully verified in the first word sequence, and the second target word is the word that has been successfully verified in the third word sequence.

[0180] The fourth determination module is used to determine the natural language text as the output text after decoding the word at the position following each word in the third word sequence based on M+1 second seed sequences to obtain the fourth word sequence.

[0181] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described text processing method is also provided. This electronic device may, but is not limited to, […]. Figure 1 The client 112 or server 112 shown in this embodiment is illustrated using an electronic device as the client 112. Further details are provided below. Figure 11 As shown, the electronic device includes a memory 1102 and a processor 1104. The memory 1102 stores a computer program, and the processor 1104 is configured to execute the steps of any of the above method embodiments via the computer program.

[0182] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0183] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0184] S1, For the input text, obtain the first word sequence used to obtain the output text. The word at the first position in the first word sequence is output by the decoder. The other M words in the first word sequence are obtained by means other than the decoder, where M is a positive integer.

[0185] S2, the decoder decodes the word at the next position of each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence. One word in the first word sequence and one first seed sequence correspond to one word in the second word sequence.

[0186] S3, the M words in the first word sequence are verified by the second word sequence. Among the M words in the first word sequence, the first target word and all words before the first target word are verified successfully, while the second target word is located after the first target word and fails to be verified.

[0187] S4, obtain the third word sequence used to obtain the output text. The word at the first position in the third word sequence belongs to the second word sequence and is the word at the next position after the first target word. The other M words in the third word sequence are obtained by means other than the decoder.

[0188] S5, using a decoder based on M+1 second seed sequences, decodes the word at the position following each word in the third word sequence to obtain the fourth word sequence. One word in the third word sequence and one second seed sequence correspond to one word in the fourth word sequence. The second seed sequence used to decode the word at the position following the third target word in the third word sequence is the same as the first seed sequence used to decode the word at the position following the second target word. The third target word and the second target word correspond to the same position in the output text.

[0189] Alternatively, as those skilled in the art will understand, Figure 11 The structure shown is for illustrative purposes only. Figure 11 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 11 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 11 The different configurations shown.

[0190] The memory 1102 can be used to store software programs and modules, such as the program instructions / modules corresponding to the text processing method and apparatus in this embodiment. The processor 1104 executes various functional applications and data processing by running the software programs and modules stored in the memory 1102, thereby implementing the aforementioned text processing method. The memory 1102 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1102 may further include memory remotely located relative to the processor 1104, and these remote memories can be connected to electronic devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1102 may be used, but is not limited to, to store information such as input text and output text. As an example, such as... Figure 11 As shown, the memory 1102 may include, but is not limited to, the first acquisition unit 1002, the first decoding unit 1004, the verification unit 1006, the second acquisition unit 1008, and the second decoding unit 1010 in the text processing device. Furthermore, it may include, but is not limited to, other module units in the text processing device, which will not be elaborated upon in this example.

[0191] Optionally, the transmission device 1106 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1106 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1106 is a radio frequency (RF) module, used for wireless communication with the Internet.

[0192] In addition, the aforementioned electronic device also includes: a display 1108 for displaying input text, output text and other information; and a connection bus 1110 for connecting various module components in the aforementioned electronic device.

[0193] In other embodiments, the aforementioned client or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer network, and any form of computing device, such as a server, client, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0194] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions provided in embodiments of this application.

[0195] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0196] It should be noted that the computer system of the electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0197] A computer system includes a Central Processing Unit (CPU), which performs various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) or loaded from RAM. ROM also stores various programs and data required for system operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output interfaces (I / O interfaces) are also connected to the bus.

[0198] The following components are connected to the input / output interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard drives; and communication sections including network interface cards such as LAN cards and modems. The communication section performs communication processing via a network such as the Internet. Drives are also connected to the input / output interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0199] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions defined in the system of this application.

[0200] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of a computer device reads computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations described above.

[0201] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store a computer program for performing the following steps:

[0202] S1, For the input text, obtain the first word sequence used to obtain the output text. The word at the first position in the first word sequence is output by the decoder. The other M words in the first word sequence are obtained by means other than the decoder, where M is a positive integer.

[0203] S2, the decoder decodes the word at the next position of each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence. One word in the first word sequence and one first seed sequence correspond to one word in the second word sequence.

[0204] S3, the M words in the first word sequence are verified by the second word sequence. Among the M words in the first word sequence, the first target word and all words before the first target word are verified successfully, while the second target word is located after the first target word and fails to be verified.

[0205] S4, obtain the third word sequence used to obtain the output text. The word at the first position in the third word sequence belongs to the second word sequence and is the word at the next position after the first target word. The other M words in the third word sequence are obtained by means other than the decoder.

[0206] S5, using a decoder based on M+1 second seed sequences, decodes the word at the position following each word in the third word sequence to obtain the fourth word sequence. One word in the third word sequence and one second seed sequence correspond to one word in the fourth word sequence. The second seed sequence used to decode the word at the position following the third target word in the third word sequence is the same as the first seed sequence used to decode the word at the position following the second target word. The third target word and the second target word correspond to the same position in the output text.

[0207] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware of an electronic device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0208] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0209] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods of the various embodiments of this application.

[0210] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0211] In the several embodiments provided in this application, it should be understood that the recorded client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0212] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0213] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0214] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A text processing method, characterized in that, include: For the input text, a first word sequence for obtaining the output text is obtained, wherein the word at the first position in the first word sequence is output by the decoder, and the other M words in the first word sequence are obtained by means other than the decoder, where M is a positive integer; The decoder decodes the word at the next position of each word in the first word sequence based on M+1 first seed sequences to obtain a second word sequence. One word in the first word sequence and one first seed sequence correspond to one word in the second word sequence. The M words in the first word sequence are verified by the second word sequence, wherein the first target word and all words before the first target word are successfully verified, and the second target word is located after the first target word and fails to be verified. Obtain a third word sequence for obtaining the output text, wherein the word at the first position in the third word sequence belongs to the second word sequence and is the word at the position following the first target word, and the other M words in the third word sequence are obtained by means other than the decoder; The decoder decodes the word one position after each word in the third word sequence based on M+1 second seed sequences to obtain the fourth word sequence. One word in the third word sequence and one second seed sequence correspond to one word in the fourth word sequence. The second seed sequence used to decode the word at the position following the third target word in the third word sequence is the same as the first seed sequence used to decode the word at the position following the second target word. The third target word and the second target word correspond to the same position in the output text.

2. The method according to claim 1, characterized in that, The step of verifying the M lexical units in the first lexical sequence using the second lexical sequence includes: The Nth second word in the first M word sequence is used sequentially to verify the Nth first word in the first M word sequence, where N is a positive integer; If a reference word that is different from the corresponding second word appears for the first time among the M first word words, the reference word word is determined to have failed the verification, the reference word word is determined to be the second target word word, and the word word one position before the reference word word among the M first word words is determined to be the first target word word.

3. The method according to claim 2, characterized in that, The step of obtaining the third word sequence for obtaining the output text includes: The second word corresponding to the reference word and the other M words obtained by means other than the decoder are determined as the third word sequence, wherein the second word corresponding to the reference word is the word at the first position in the third word sequence; Before decoding the word at the position following each word in the third word sequence based on M+1 second seed sequences using the decoder to obtain the fourth word sequence, the method further includes: Obtain the M+1 second seed sequences, wherein the reference seed sequence is the first second seed sequence among the M+1 second seed sequences, and the reference seed sequence is the first seed sequence among the M+1 first seed sequences used to generate the reference word.

4. The method according to claim 1, characterized in that, After the method decodes the word at the next position after each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence, the method further includes: The M words in the first word sequence are verified by the second word sequence, wherein all words in the M words of the first word sequence are successfully verified. Obtain a fifth word sequence for obtaining the output text, wherein the word at the first position in the fifth word sequence is the last word in the M+1 words of the second word sequence, and the other M words in the fifth word sequence are obtained by means other than the decoder; The decoder decodes the word at the position following each word in the fifth word sequence based on M+1 third seed sequences to obtain the sixth word sequence. A word in the fifth word sequence and a third seed sequence correspond to a word in the sixth word sequence.

5. The method according to claim 1, characterized in that, The step of decoding the word at the position following each word in the first word sequence based on M+1 first seed sequences using the decoder to obtain the second word sequence includes: During the decoding process of the word at the position following the Qth word in the first word sequence by the decoder, regression prediction is performed based on the first Q words in the first word sequence to obtain P output words. Each output word corresponds to a probability score, which is used to represent the probability that the output word is the word at the position following the Qth word. Q and P are positive integers. The P probability scores corresponding to the P output words are integrated with the first seed sequence corresponding to the next position of the Qth word to obtain P target probability scores. The output word with the highest target probability score among the P target probability scores is determined as the second word corresponding to the next position of each word in the first word sequence, and the second word sequence includes the second word.

6. The method according to claim 5, characterized in that, The regression prediction based on the first Q words in the first word sequence yields P output words, including: Regression prediction is performed on the first Q words in the first word sequence to obtain multiple initial output words; From the plurality of initial output words, determine the K output words with the highest probability scores, where K is a positive integer; The sum of the probability scores of the K output words is determined from the P output words that are greater than or equal to a preset threshold.

7. The method according to any one of claims 1 to 6, characterized in that, include: For the input text, obtaining the first word sequence used to obtain the output text includes: The input text is segmented to obtain multiple keywords; Perform a first sequence format conversion on the multiple keywords to obtain multiple word tensors after conversion; The plurality of lexical tensors are subjected to a first decoding process to obtain the lexical element at the first position in the first lexical element sequence, and the plurality of lexical tensors are subjected to a second decoding process to obtain the other M lexical elements in the first lexical element sequence. After the method decodes the word at the position following each word in the third word sequence based on M+1 second seed sequences to obtain the fourth word sequence, the method further includes: The first target word in the first word sequence, the first target word in the first word sequence, and the second target word in the third word sequence are subjected to a second sequence format conversion to obtain the converted natural language text. The first target word is the word that has been successfully verified in the first word sequence, and the second target word is the word that has been successfully verified in the third word sequence. The natural language text is determined as the output text.

8. A text processing device, characterized in that, include: The first acquisition unit is used to acquire, for input text, a first word sequence for obtaining output text. The word at the first position in the first word sequence is output by the decoder, and the other M words in the first word sequence are obtained by means other than the decoder, where M is a positive integer. The first decoding unit is used to decode the word at the next position of each word in the first word sequence based on M+1 first seed sequences to obtain the second word sequence. One word in the first word sequence and one first seed sequence correspond to one word in the second word sequence. The verification unit is used to verify M words in the first word sequence using the second word sequence. Among the M words in the first word sequence, the first target word and all words before the first target word are successfully verified, while the second target word is located after the first target word and fails to be verified. The second acquisition unit is used to acquire the third word sequence for obtaining the output text. The word at the first position in the third word sequence belongs to the second word sequence and is the word at the next position after the first target word. The other M words in the third word sequence are obtained by means other than the decoder. The second decoding unit is used to decode the word at the next position of each word in the third word sequence based on M+1 second seed sequences to obtain the fourth word sequence. One word in the third word sequence and one second seed sequence correspond to one word in the fourth word sequence. The second seed sequence used to decode the word at the position following the third target word in the third word sequence is the same as the first seed sequence used to decode the word at the position following the second target word. The third target word and the second target word correspond to the same position in the output text.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program is executed by an electronic device to perform the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

11. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 7 through the computer program.

Images