Text processing method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application disclose a text processing method and device, equipment and medium. The method comprises the following steps: extracting a feature vector of each character included in a to-be-identified text, wherein the feature vector of each character comprises context features of the character; then, generating a feature vector sequence corresponding to the to-be-identified text according to the feature vector of each character, and further generating features of a text segment included in the to-be-identified text according to the feature vector sequence, wherein the text segment comprises a plurality of characters; and finally, identifying fluency of the text segment according to the features of the text segment. The technical solution of the present application improves the accuracy of text fluency identification.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and more specifically, to a text processing method, a text processing apparatus, an electronic device, and a computer-readable medium. Background Technology

[0002] On internet platforms, creators produce a massive amount of text every day. However, this vast amount of text often includes incoherent sentences, which greatly impacts the reading experience. Therefore, it is necessary to identify the fluency of the text.

[0003] However, natural language is highly variable, and the reasons for errors are also very diverse. For example, text incoherence may be caused by superficial errors such as misspellings, extra words, missing words, or word order inversion, or by deeper grammatical errors such as misuse of prepositions or improper collocation, or even by deeper logical errors such as the confusion between "ba" and "bei" sentences. This greatly increases the difficulty of recognizing text fluency.

[0004] Therefore, improving the accuracy of text fluency recognition is an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, embodiments of this application provide a text processing method, apparatus, device, and medium, thereby improving the accuracy of text fluency recognition to at least a certain extent.

[0006] According to one aspect of the embodiments of this application, this application provides a text processing method, the method comprising:

[0007] Extract the feature vector of each character in the text to be identified, wherein the feature vector of each character includes the context features of each character;

[0008] Based on the feature vector of each character, generate a sequence of feature vectors corresponding to the text to be identified;

[0009] Based on the feature vector sequence, features of text segments included in the text to be identified are generated, wherein the text segments include multiple characters;

[0010] Based on the characteristics of the text fragment, the fluency of the text fragment is identified.

[0011] According to one aspect of the embodiments of this application, this application provides a text processing apparatus, the apparatus comprising:

[0012] The extraction module is configured to extract the feature vector of each character in the text to be recognized, wherein the feature vector of each character includes the context features of each character;

[0013] The sequence generation module is configured to generate a sequence of feature vectors corresponding to the text to be identified based on the feature vector of each character.

[0014] The feature generation module is configured to generate features of text segments included in the text to be identified based on the feature vector sequence, wherein the text segments include multiple characters;

[0015] The recognition module is configured to identify the fluency of the text segment based on its features.

[0016] In one embodiment of this application, based on the foregoing scheme, the extraction module includes:

[0017] The context feature extraction module is configured to extract the context features of each character in the text to be recognized;

[0018] The encoding module is configured to encode each character to obtain the encoding features of each character;

[0019] The extraction submodule is configured to generate a feature vector for each character based on the contextual features and encoding features of each character.

[0020] In one embodiment of this application, based on the foregoing scheme, the context feature extraction module includes:

[0021] The context feature and context feature generation module is configured to generate context features for each character based on the relationship between each character and its context in the text to be identified, and to generate context features for each character based on the relationship between each character and its context in the text to be identified;

[0022] The context feature determination module is configured to determine the context features of each character based on the preceding and following context features of each character.

[0023] In one embodiment of this application, based on the foregoing scheme, the extraction submodule is specifically configured as follows:

[0024] The context features and encoding features of each character are merged to obtain the merged features of each character;

[0025] Based on the merging features of each character, a merging feature sequence corresponding to the text to be identified is generated, wherein the merging feature sequence includes the merging features of each character;

[0026] The association between each character and other characters in the text to be identified, excluding each character, is calculated based on the merged feature sequence.

[0027] The feature vector of each character is generated based on the relationship between each character and the other characters.

[0028] In one embodiment of this application, based on the foregoing scheme, the feature generation module includes:

[0029] The vector acquisition module is configured to acquire a feature vector subsequence with a length matching a specified length from the feature vector sequence;

[0030] The feature generation submodule is configured to generate features of the text segment corresponding to the feature vector subsequence based on the feature vectors included in the feature vector subsequence.

[0031] In one embodiment of this application, based on the foregoing scheme, the vector acquisition module is specifically configured as follows:

[0032] Feature vector subsequences are obtained from the feature vector sequence using multiple strategies;

[0033] In the process of obtaining a feature vector subsequence from the feature vector sequence using any strategy, a specified length corresponding to any strategy is obtained. Starting from each feature vector included in the feature vector sequence, a feature vector subsequence with a length matching the specified length corresponding to any strategy is obtained from the feature vector sequence according to a set direction. The specified lengths are different for different strategies.

[0034] In one embodiment of this application, based on the foregoing scheme, the recognition module is specifically configured to: input the features of the text segment into a pre-trained fluency recognition model, so as to recognize the fluency of the text segment through the fluency recognition model;

[0035] The feature generation module is specifically configured to: obtain the feature vector of each character included in the text segment from the feature vector sequence; process the feature vector of each character included in the text segment through the fluency recognition model to obtain the weight of each character included in the text segment; and fuse the feature vectors of multiple characters included in the text segment according to the obtained weights to obtain the features of the text segment.

[0036] In one embodiment of this application, based on the aforementioned scheme, the extraction module is specifically configured to: extract the feature vector of each character included in the text to be identified based on the feature extraction model obtained through unsupervised training;

[0037] The recognition module is specifically configured to process the features of the text segment based on a fluency recognition model obtained through supervised training, so as to obtain the fluency of the text segment.

[0038] In one embodiment of this application, based on the foregoing solution, the apparatus further includes:

[0039] The sample feature extraction module is configured to acquire sample text and the annotation information of the sample text, and extract the context features of each character included in the sample text through the feature extraction model;

[0040] The sample text fragment feature generation module is configured to generate features of the sample text fragments included in the sample text based on the context features of each character included in the sample text; the sample text fragments include multiple characters.

[0041] The sample recognition module is configured to analyze the features of the sample text segment based on the fluency recognition model to obtain the fluency recognition result of the sample text segment;

[0042] The adjustment module is configured to adjust the model parameters of the fluency recognition model based on the fluency recognition result of the sample text fragment and the annotation information.

[0043] In one embodiment of this application, based on the foregoing scheme, the adjustment module includes:

[0044] The quantity determination module is configured to determine the number of annotated characters included in the sample text segment based on the annotation information;

[0045] The adjustment submodule is configured to adjust the model parameters of the fluency recognition model based on the number of annotated characters and the fluency recognition result of the sample text fragment.

[0046] In one embodiment of this application, based on the foregoing scheme, the adjustment submodule is specifically configured as follows:

[0047] Determine the ratio between the number of annotated characters and the length of the sample text segment;

[0048] Based on the fluency recognition results of the sample text fragment, a probability value representing the fluency of the sample text fragment is generated;

[0049] The loss of the fluency recognition model is calculated based on the difference between the probability value and the ratio.

[0050] The model parameters of the fluency recognition model are adjusted based on the loss of the fluency recognition model.

[0051] According to one aspect of the embodiments of this application, an electronic device is provided, including:

[0052] One or more processors;

[0053] A storage device for storing one or more computer programs that, when executed by one or more processors, cause the electronic device to perform the text processing method as described above.

[0054] According to one aspect of the embodiments of this application, an embodiment of this application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor of an electronic device, causes the electronic device to implement the text processing method as described above.

[0055] According to one aspect of the embodiments of this application, the embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the text processing method as described above.

[0056] In the technical solution provided by the embodiments of this application: First, the feature vector of each character in the text to be identified is extracted, wherein the feature vector of each character includes the context features of the character, so that the feature vector of each character can better express the meaning of the character in the text to be identified; then, based on the feature vector of each character, a feature vector sequence corresponding to the text to be identified is generated, and then the features of the text fragments included in the text to be identified are generated based on the feature vector sequence, wherein the text fragments include multiple characters; then, based on the features of the text fragments, the fluency of the text fragments is identified. That is to say, the fluency of the text fragments included in the text to be identified can be identified in the embodiments of this application. Since text incoherence is usually caused by an error in a fragment, fluency identification based on text fragments can improve the accuracy of fluency identification. Furthermore, the features of the text fragments are generated based on feature vectors including context features, thus further improving the accuracy of fluency identification.

[0057] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0059] Figure 1 This is a schematic diagram illustrating an exemplary implementation environment of the technical solutions in the embodiments of this application;

[0060] Figure 2 This is a schematic diagram of another exemplary implementation environment of the technical solution of this application embodiment;

[0061] Figure 3 This is a flowchart illustrating a text processing method in an exemplary embodiment of this application;

[0062] Figure 4 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0063] Figure 5 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0064] Figure 6 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0065] Figure 7 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0066] Figure 8 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0067] Figure 9 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0068] Figure 10 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0069] Figure 11 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0070] Figure 12 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0071] Figure 13 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0072] Figure 14 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application;

[0073] Figure 15 This is a schematic diagram of the overall architecture of the fluency recognition model shown in another exemplary embodiment of this application;

[0074] Figure 16 This is a flowchart illustrating a text processing method in an exemplary embodiment of this application;

[0075] Figure 17 This is a schematic diagram illustrating the compositional architecture of a converter-based bidirectional language model, as shown in an exemplary embodiment of this application.

[0076] Figure 18 This is a schematic diagram illustrating feature vector extraction, as shown in an exemplary embodiment of this application.

[0077] Figure 19 This is a block diagram of a text processing apparatus according to an embodiment of this application;

[0078] Figure 20 This is a schematic diagram of the structure of a computer system suitable for implementing the electronic devices of the present application embodiments. Detailed Implementation

[0079] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments identical to those of this application. Rather, they are merely examples of apparatuses and methods identical to some aspects of this application as detailed in the appended claims.

[0080] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0081] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0082] It should be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0083] The technical solutions of the embodiments of this application relate to artificial intelligence (AI). Before introducing the technical solutions of the embodiments of this application, artificial intelligence will be briefly introduced first.

[0084] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0085] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0086] Natural Language Processing (NLP) is an important area within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close connection with linguistic research. NLP technologies typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0087] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0088] The technical solutions of this application specifically relate to machine learning and natural language processing technologies in artificial intelligence, specifically text processing based on machine learning and natural language processing technologies. The technical solutions of this application are described in detail below:

[0089] On internet platforms, creators generate a massive amount of text every day, especially in the era of self-media, where creators produce millions of articles daily for distribution platforms to be recommended to hundreds of millions of users. Among this vast amount of text, there are often incoherent sentences, which greatly impacts the reading experience. Therefore, it is necessary to identify the fluency of the text in order to correct it.

[0090] However, natural languages ​​are incredibly diverse, and the reasons for errors vary greatly. For example, compared to English, Chinese expressions are richer and more varied, and its grammatical and syntactic structures are more flexible and varied. Errors can arise from a wide range of sources, including superficial errors such as misspellings, extra or missing characters, and word order inversions; deeper grammatical errors such as misuse of prepositions and improper collocations; and more profound logical errors such as the confusion between plural and nonverbal sentences. This significantly increases the difficulty of text fluency recognition. Therefore, improving the accuracy of text fluency recognition is a pressing issue. Based on this, embodiments of this application provide a text processing method, a text processing device, an electronic device, and a computer-readable medium that can improve the accuracy of text fluency recognition.

[0091] Please see Figure 1 , Figure 1 This is a schematic diagram of an implementation environment related to this application. The implementation environment includes a terminal device 110 and a server 120. The terminal device 110 and the server 120 communicate via a wired or wireless network. The terminal device 110 can upload its own data to the server 120 and can also retrieve data from the server 120.

[0092] Among them, terminal equipment 110 may include, but is not limited to, smartphones, tablets, laptops, computers, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc.

[0093] Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This document does not impose any restrictions on this.

[0094] It should be noted that, Figure 1 The number of terminal devices 110 and servers 120 is merely illustrative; any number of terminal devices 110 and servers 120 can be used as needed.

[0095] In an exemplary embodiment, the text processing method provided in the embodiments of this application can be executed by a terminal device 110.

[0096] For example, the terminal device 110 can acquire the text to be recognized and extract the feature vector of each character in the text to be recognized. The feature vector of each character includes the context features of the character, so that the feature vector of each character can better express the meaning of the character in the text to be recognized. Then, based on the feature vector of each character, a feature vector sequence corresponding to the text to be recognized is generated. Then, based on the feature vector sequence, the features of the text fragments included in the text to be recognized are generated. The text fragments include multiple characters. Then, based on the features of the text fragments, the fluency of the text fragments is identified. That is, in this embodiment, the fluency of the text fragments included in the text to be recognized can be identified. Since text incoherence is usually caused by a paragraph, identifying fluency based on text fragments can improve the accuracy of fluency recognition. Compared with the method of identifying fluency based on a single character, it can reduce the probability of missed recognition. Furthermore, the features of the text fragments are generated based on feature vectors including context features, so the accuracy of fluency recognition can be further improved.

[0097] In another exemplary embodiment, server 120 may have functions similar to terminal device 110 to perform the text processing method provided in this application embodiment. For example, server 120 may acquire text to be recognized and extract feature vectors for each character included in the text to be recognized, wherein the feature vector of each character includes the contextual features of the character, making the feature vector of each character better express the meaning of the character in the text to be recognized; then, based on the feature vectors of each character, a feature vector sequence corresponding to the text to be recognized is generated, and then features of text segments included in the text to be recognized are generated based on the feature vector sequence, wherein the text segments include multiple characters; finally, the fluency of the text segments is identified based on the features of the text segments.

[0098] In another exemplary embodiment, the terminal device 110 and the server 120 may also jointly execute the text processing method provided in the embodiments of this application. For example, the terminal device 110 may acquire the text to be recognized and upload it to the server 120; the server 120 may extract the feature vector of each character in the text to be recognized, generate a feature vector sequence corresponding to the text to be recognized based on the feature vector of each character, generate features of the text segments included in the text to be recognized based on the feature vector sequence, identify the fluency of the text segments based on the features of the text segments, and transmit the fluency identification result of the text segments to the terminal device 110.

[0099] It should be noted that the text processing method provided in the embodiments of this application can be applied to different application scenarios. In an exemplary embodiment, the text processing method can be applied to a content distribution scenario, see [link to example]. Figure 2 , Figure 2 This is a schematic diagram of a specific implementation environment involved in this application. This environment includes a text provider 210, a content distribution platform 220, and a text consumer 230. After obtaining the text created by the creator, the text provider 210 transmits the text as the text to be identified to the content distribution platform 220. The content distribution platform 220 extracts the feature vector of each character in the text to be identified, generates a feature vector sequence corresponding to the text to be identified based on the feature vector sequence, and then generates features of the text segments included in the text to be identified based on the features of the text segments. The fluency of the text segments is then identified based on the features of the text segments, and the fluency identification result is transmitted to the text provider 210. The text provider 210 displays the fluency identification result of the text segments in the text to be identified, allowing the creator to modify any incoherent parts of the text to be identified. The modified text is then transmitted to the content distribution platform 220 through the text provider 210. The content distribution platform 220 distributes the modified text to the text consumer 230 for users to read, thereby improving the user's reading experience.

[0100] It should be noted that in this application, when the text to be identified and other data related to the object are applied to specific products or technologies, the permission or consent of the object is obtained, and the extraction, use and processing of the relevant data comply with local security standards and local laws and regulations.

[0101] The following details the various implementation details of the technical solutions in the embodiments of this application:

[0102] Please see Figure 3 , Figure 3 This is a flowchart illustrating a text processing method according to an embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0103] like Figure 3 As shown, this text processing method includes at least steps S310 to S340, which are described in detail below:

[0104] Step S310: Extract the feature vector of each character in the text to be recognized. The feature vector of each character includes the context features of each character.

[0105] It should be noted that the text to be identified refers to the text whose fluency needs to be determined. It can be an article, a paragraph, a sentence, or a phrase, etc. The text to be identified can include, but is not limited to, single-language texts such as Chinese text, English text, and Korean text. Alternatively, the text to be identified can also include multi-language texts, such as mixed Chinese and English texts or mixed Chinese and Korean texts.

[0106] In natural language, the same word can have different meanings in different contexts. For example, in the sentence "The kitten is playing with a ball, it's so happy," "it" refers to the kitten, while in the sentence "This puppy can't talk," "it" refers to the puppy. Therefore, to improve the accuracy of fluency recognition, in this embodiment, after obtaining the text to be recognized, the contextual features of each word in the text are extracted to obtain a feature vector that includes the contextual features of each word.

[0107] The context features of each character are used to characterize the context of the word. These features can be any features that can characterize the context of the word. For example, they can include, but are not limited to, features determined based on at least one of the following: the context of each character in its context, the part of speech of each character in its context, the probability of the character predicted based on its preceding context, the probability of the character predicted based on its following context, the preceding context of each character, and the following context of each character.

[0108] Optionally, to improve the accuracy of feature extraction, the text to be recognized can be input into a machine learning model to extract the contextual features of each character in the text. The machine learning model used to extract contextual features can be an unsupervised training model, such as a language model; or it can be a supervised training model.

[0109] For a language model, it is a model obtained through language abstract mathematical modeling based on objective language facts. Language models include, but are not limited to, statistical language models and neural language models. Among them, a statistical language model is a mathematical model that starts from the perspective of probability statistics and solves the characteristics of natural language related to context. For example, an n-gram language model. A neural language model is a model obtained by modeling the probability distribution of sentences through a neural network model on the basis of a statistical language model. For example, a feedforward neural network language model, a recurrent neural network language model, a recurrent neural network language model with long short-term memory, a language model based on Transformer, etc. A language model includes a unidirectional language model, that is, a language model that makes predictions from front to back according to the text reading order (i.e., forward prediction), or a language model that makes predictions from back to front according to the text reading order (i.e., backward prediction); or, a language model includes a bidirectional language model, that is, a language model that includes forward prediction and backward prediction. In the embodiments of the present application, the specific type of the language model is not limited.

[0110] Step S320: Generate a feature vector sequence corresponding to the text to be recognized according to the feature vectors of each character.

[0111] After obtaining the feature vectors of each character included in the text to be recognized, a feature vector sequence corresponding to the text to be recognized can be generated according to the feature vectors of each character included in the text to be recognized.

[0112] Optionally, the feature vectors corresponding to the characters included in the text to be recognized can be sorted according to the positions of the characters included in the text to be recognized in the text to be recognized, so as to obtain the feature vector sequence corresponding to the text to be recognized. For example, assume that the text to be recognized is "The weather is good today". Among them, the feature vector of "今" is c1, the feature vector of "日" is c2, the feature vector of "天" is c3, the feature vector of "气" is c4, and the feature vector of "好" is c5. Then the feature vector sequence of the text to be recognized is: c1, c2, c3, c4, c5.

[0113] Step S330: Generate the features of the text fragments included in the text to be recognized according to the feature vector sequence. The text fragment includes multiple characters.

[0114] A text fragment is a part of the text, which includes multiple characters. For example, 3 characters, 4 characters, etc.

[0115] A text may be unsmooth due to typos, extra characters, missing characters, disorder, reversed word order, etc. Generally, the error occurs in a fragment. For example, as shown in Table 1, Table 1 lists common unsmooth types and corresponding examples.

[0116]

[0117]

[0118] Table 1

[0119] Therefore, in order to improve the accuracy of fluency recognition and reduce the probability of missed recognition, in the embodiments of this application, text segments can be determined from the text to be recognized, and features of the text segments can be generated according to the feature vector sequence corresponding to the text to be recognized.

[0120] Step S340: Identify the fluency of the text fragment based on its characteristics.

[0121] After generating the features of the text segments included in the text to be identified, the fluency of the text segments can be identified based on these features.

[0122] Optionally, to improve the accuracy of fluency recognition, the features of the text fragment can be input into a machine learning model. The machine learning model analyzes the features of the text fragment to obtain its fluency. The machine learning model used for fluency recognition can be an unsupervised or supervised training model. In this embodiment, the specific type of machine learning model used for fluency recognition is not limited.

[0123] Optionally, after obtaining the fluency recognition results of the text segments included in the text to be recognized, the fluency recognition results of the text segments can be displayed so that the text provider is aware of the fluency recognition results. To make it clearer to the text provider which segments are not fluent and to facilitate correction, when displaying the fluency recognition results of the text segments, the text to be recognized can be displayed, and the text segments identified as not fluent can be distinguished. The methods for distinguishing these segments include, but are not limited to, increasing font size, changing text color, adding underlines, changing font, and changing text background. In one example, assuming the text to be recognized is "2022 Popular Mobile Games Review", and the fluent segment is "Review", then the color of "Review" can be set to red, and the color of other text can be set to black, making the fluent segment distinguishable from the other text and facilitating correction by the text provider.

[0124] Optionally, after obtaining the fluency recognition result of the text segments included in the text to be recognized, if the fluency recognition result is not fluent, the fluent text segments can be automatically corrected to improve the reading experience.

[0125] exist Figure 3In the illustrated embodiment, feature vectors for each character in the text to be identified are first extracted. Each character's feature vector includes its contextual features, making the feature vectors of each character better express the meaning of the character in the text to be identified. Then, based on the feature vectors of each character, a sequence of feature vectors corresponding to the text to be identified is generated. Subsequently, features of text segments in the text to be identified are generated based on the feature vector sequence, where each text segment includes multiple characters. Then, based on the features of the text segments, the fluency of the text segments is identified. In other words, this embodiment can identify the fluency of text segments in the text to be identified. Since text incoherence is usually caused by a paragraph, identifying fluency based on text segments can improve the accuracy of fluency identification. Furthermore, since the features of the text segments are generated based on feature vectors including contextual features, the accuracy of fluency identification can be further improved.

[0126] In one exemplary embodiment, see Figure 4 , Figure 4 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0127] like Figure 4 As shown, the text processing method includes steps S410-S430 and steps S320-S340, wherein the detailed description of steps S410-S430 is as follows:

[0128] Step S410: Extract the contextual features of each character in the text to be identified.

[0129] In the embodiments of this application, the contextual features of each character can be extracted from the text to be identified.

[0130] Step S420: Encode each character to obtain the encoding features of each character.

[0131] The encoding features of each character are used to characterize the encoding information of each character, and their forms include, but are not limited to, vectors, matrices, and numerical values.

[0132] In this process, each character in the text to be recognized is encoded, and the specific method for obtaining the encoding features of each character can be flexibly set according to actual needs.

[0133] For example, in one alternative approach, the encoded features of each character in the text to be recognized can be obtained by querying a pre-defined encoding feature table (e.g., a character vector table).

[0134] Alternatively, in another optional approach, each character in the text to be identified can be encoded using preset encoding rules, mapping algorithms, etc., to obtain the encoded features of each character.

[0135] Alternatively, in another optional approach, each character in the text to be recognized can be input into a machine learning model. The machine learning model then processes each character to obtain its encoded features. The specific type of machine learning model used to obtain these encoded features can be flexibly set according to actual needs. Since the BERT (Bidirectional Encoder Representation from Transformers) model extracts comprehensive encoded features, it can be used to process each character in the text to obtain its encoded features.

[0136] Step S430: Generate a feature vector for each character based on its contextual features and encoding features.

[0137] After obtaining the top-bottom features and encoding features of each character in the text to be recognized, a feature vector for each character can be generated based on the top-bottom features and encoding features of each character. This allows the feature vector of each character to represent both the features of the character itself and the contextual features of the character.

[0138] It should be noted that, Figure 4 The specific implementation details of steps S320-S340 shown can be found in [reference]. Figure 3 Steps S320-S340 shown will not be repeated here.

[0139] exist Figure 4 In the illustrated embodiment, a feature vector for each character is generated based on the contextual and encoding features of each character in the text to be identified. This allows the feature vector to more accurately represent the true meaning of the character in the text to be identified, thereby improving the accuracy of subsequent fluency recognition.

[0140] In one exemplary embodiment, see Figure 5 , Figure 5 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1is executed by server 120 in the illustrated implementation environment, or alternatively, can be jointly executed by Figure 1 terminal device 110 and server 120 in the illustrated implementation environment.

[0141] As Figure 5 shown, the text processing method includes steps S510 - step S520, steps S420 - step S430, and steps S320 - step S340. Among them, the detailed introduction of steps S510 - step S520 is as follows:

[0142] Step S510, generate the upstream feature of each word according to the relationship between each word and the upstream text of each word in the text to be recognized, and generate the downstream feature of each word according to the relationship between each word and the downstream text of each word in the text to be recognized.

[0143] It should be noted that the upstream text of each word in the text to be recognized refers to the text located before each word in the text to be recognized. The downstream text of each word in the text to be recognized refers to the text located after each word in the text to be recognized. For example, if the text to be recognized is "2022 popular mobile games inventory broadcast", then for the word "year", its upstream includes "2022", and its downstream includes "popular mobile games inventory broadcast".

[0144] In the embodiments of the present application, the upstream feature of each word can be generated according to the relationship between each word and the upstream text of each word in the text to be recognized, and the downstream feature of each word can be generated according to the relationship between each word and the downstream text of each word in the text to be recognized.

[0145] Among them, the specific manner of generating the upstream feature of each word can be flexibly set according to actual needs. For example, the context of the word can be determined according to the upstream text of each word in the text to be recognized, and the upstream feature of the word can be generated according to the context; or the probability of the word can be predicted according to the upstream text of each word in the text to be recognized, and the upstream feature of the word can be generated according to the probability; or the upstream feature of the word can be generated according to the representation vector corresponding to the upstream text of each word. It should be noted that the representation vector corresponding to the upstream text is used to represent the text corresponding to the upstream.

[0146] Correspondingly, the specific manner of generating the downstream feature of each word can be flexibly set according to actual needs. For example, the context of the word can be determined according to the downstream text of each word in the text to be recognized, and the downstream feature of the word can be generated according to the context; or the probability of the word can be predicted according to the downstream text of each word in the text to be recognized, and the downstream feature of the word can be generated according to the probability; or the downstream feature of the word can be generated according to the representation vector corresponding to the downstream text of each word.

[0147] Optionally, to enhance the comprehensiveness of the above-mentioned features and the following features, for each character included in the text to be recognized, all the characters before this character can be determined from the text to be recognized, and the determined characters are used as the above context of this character. According to the relationship between this character and its above context, the above context feature is determined; or, for each character included in the text to be recognized, all the characters after this character can be determined from the text to be recognized, and the determined characters are used as the following context of this character. According to the relationship between this character and its following context, the following context feature is determined. For example, if the text to be recognized is "2022 Popular Mobile Games Inventory Broadcast", for the character "year", the above context feature of "year" can be determined according to the relationship between "year" and "2022", and the following context feature of "year" can be determined according to the relationship between "year" and "Popular Mobile Games Inventory Broadcast".

[0148] Alternatively, since the greater the distance between two characters, the smaller their关联性 (it seems there is a typo here, assuming it might be "correlation"), in order to save computing resources, for each character included in the text to be recognized, the characters before this character and with a distance less than the first distance threshold from this character can be determined from the text to be recognized, and the determined characters are used as the above context of this character. According to the relationship between this character and its above context, the above context feature is determined; or, for each character included in the text to be recognized, the characters after this character and with a distance less than the second distance threshold from this character can be determined from the text to be recognized, and the determined characters are used as the following context of this character. According to the relationship between this character and its following context, the following context feature is determined. The first distance threshold and the second distance threshold can be flexibly set according to actual needs. The first distance threshold and the second distance threshold can be equal or unequal. For example, the first distance threshold and the second distance threshold can both be set to 1, that is, the above context of each character is the previous character of this character, and the following context of each character is the next character of this character. In an example, if the text to be recognized is "2022 Popular Mobile Games Inventory Broadcast" and the first distance threshold and the second distance threshold are both 1, for the character "hot", the representation vector of "year" can be used as the above context feature of "hot", and the representation vector of "door" can be used as the following context feature of "hot".

[0149] Step S520, determine the context feature of each character according to the above context feature and the following context feature of each character.

[0150] After obtaining the above context feature and the following context feature of each character, a feature vector of each character can be generated, so that the feature vector of each character includes the above context feature and the following context feature of this character.

[0151] Optionally, the above context feature and the following context feature of each character can be concatenated to obtain the context feature of this character, or the above context feature and the following context feature of each character can be added to obtain the context feature of this character.

[0152] In addition, Figure 5 In the illustrated embodiment, the contextual features of each character include the preceding and following features of the character. In other embodiments, the contextual features of each character may include only the preceding or following features of the character.

[0153] It should be noted that, Figure 5 The specific implementation details of steps S320-S340 shown can be found in [reference]. Figure 3 Steps S320-S340 are shown. Figure 5 The specific implementation details of steps S420-S430 shown can be found in [reference]. Figure 4 Steps S420-S430 shown are not repeated here.

[0154] exist Figure 5 In the illustrated embodiment, a feature vector for each character is generated based on the contextual and encoding features of each character in the text to be identified. This allows the feature vector to more accurately represent the true meaning of the character in the text to be identified, thereby improving the accuracy of subsequent fluency recognition.

[0155] In one exemplary embodiment, see Figure 6 , Figure 6 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0156] like Figure 6 As shown, the text processing method includes steps S410-S420, S610-S640, and S320-S340. The detailed descriptions of steps S610-S640 are as follows:

[0157] Step S610: Merge the context features and encoding features of each character to obtain the merged features of each character.

[0158] The merged feature of each character is obtained by merging its context features and encoding features. The specific merging method can be flexibly set according to actual needs. For example, the context features and encoding features of each character can be concatenated to obtain the merged feature of the character, or the context features and encoding features of each character can be added together to obtain the merged feature of the character.

[0159] Step S620: Generate a combined feature sequence corresponding to the text to be recognized according to the combined features of each character. The combined feature sequence includes the combined features of each character.

[0160] The combined feature sequence includes the combined features corresponding to each character included in the text to be recognized.

[0161] Optionally, the combined features corresponding to the characters included in the text to be recognized can be sorted according to the positions of each character included in the text to be recognized, so as to obtain the combined feature sequence. For example, assume the text to be recognized is "The weather is good today". Among them, the combined feature of "今" is b1, the combined feature of "日" is b2, the combined feature of "天" is b3, the combined feature of "气" is b4, and the combined feature of "好" is b5. Then the combined feature sequence of the text to be recognized is: b1, b2, b3, b4, b5.

[0162] Step S630: Calculate the correlation relationship between each character and the other characters except each character in the text to be recognized according to the combined feature sequence.

[0163] After obtaining the combined feature sequence, the combined feature sequence can be analyzed to obtain the correlation relationship between each character in the text to be recognized and the other characters except this character in the text to be recognized.

[0164] Step S640: Generate a feature vector for each character according to the correlation relationship between each character and the other characters.

[0165] After obtaining the correlation relationship between each character and the other characters in the text to be recognized, a feature vector between each character can be generated according to the correlation relationship between each character and the other characters, so that the generated feature vector has context awareness.

[0166] Optionally, the combined feature sequence can be input into a machine learning model, and the machine learning model processes the combined feature sequence to obtain the correlation relationship between each character and the other characters, and generates a feature vector for each character. Among them, the machine learning models used to process the combined feature sequence include but are not limited to the BERT model, the Convolutional Neural Network (CNN) model, the Recurrent Neural Network (RNN) model, etc.

[0167] It should be noted that Figure 6 The specific implementation details of the steps S320 - S340 shown can be referred to Figure 3 the steps S320 - S340 shown, Figure 6 The specific implementation details of the steps S410 - S420 shown can be referred to Figure 4Steps S410-S420 shown will not be repeated here.

[0168] exist Figure 6 In the illustrated embodiment, the context features and encoding features of each character are first merged to obtain the merged features of each character. Based on the merged features of each character, a merged feature sequence corresponding to the text to be recognized is generated, which includes the merged features of each character. Then, the association between each character and other characters in the text to be recognized is calculated based on the merged feature sequence. Based on the association between each character and other characters, a feature vector of each character is generated. This allows the feature vector to more accurately perceive the context of each character, thereby more accurately representing the true meaning of each character and improving the accuracy of subsequent fluency recognition.

[0169] In one exemplary embodiment, see Figure 7 , Figure 7 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0170] like Figure 7 As shown, the text processing method includes steps S310-S320, steps S710-S730, and step S340. The detailed descriptions of steps S710-S730 are as follows:

[0171] Step S710: Obtain the feature vector of each word included in the text segment from the feature vector sequence.

[0172] The feature vector sequence includes the feature vector of each character in the text to be identified. The text segment is determined from the text to be identified. Therefore, the feature vector of each character in the text segment can be obtained from the feature vector sequence.

[0173] Step S720: Determine the weight of each character in the text segment based on the feature vector of each character included in the text segment.

[0174] Weights are used to characterize the importance of fluency recognition in a text segment. The weight of a character can be determined based on the feature vector of each character included in the text segment.

[0175] The specific method for determining the weights can be flexibly set according to actual needs. Optionally, the weight of each character in the text segment can be calculated according to a preset function. The weight of each character can be linearly related to the feature vector of each character, or the weight of each character can be non-linearly related to the feature vector.

[0176] Step S730: Based on the determined weights, the feature vectors of multiple characters included in the text segment are fused to obtain the features of the text segment.

[0177] After determining the weight and feature vector of each character in the text segment, the feature vectors of multiple characters in the text segment can be fused according to the determined weights to obtain the features of the text segment.

[0178] Optionally, the feature vectors of multiple characters included in the text segment can be weighted and summed according to the determined weights to obtain the features of the text segment. For example, the features of the text segment can be calculated using the following formula:

[0179] V window =1V token_1 +2V token_2 +…+ w V token_w

[0180] Among them, V window Let a1 be the weight of the first character in the text segment to be identified, a2 be the weight of the second character in the text segment, and a3 be the weight of the first character in the text segment. w Let w be the weight of the w-th character included in the text segment, where w is the length of the text segment.

[0181] It should be noted that, Figure 7 The specific implementation details of steps S310-S320 and S340 shown can be found in [reference]. Figure 3 Steps S310-S320 and S340 shown will not be repeated here.

[0182] exist Figure 7 In the illustrated embodiment, the feature vector of each character included in the text segment is first obtained from the feature vector sequence. Then, based on the feature vector of each character included in the text segment, the weight of each character included in the text segment is determined. Then, based on the determined weight, the feature vectors of multiple characters included in the text segment are fused to obtain the features of the text segment, thereby improving the accuracy of subsequent text segment fluency recognition.

[0183] In one exemplary embodiment, see Figure 8 , Figure 8This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0184] like Figure 8 As shown, the text processing method includes steps S310-S320, S710, S810, S730, and S820. Detailed descriptions of steps S810 and S820 are as follows:

[0185] Step S820: Input the features of the text segment into the pre-trained fluency recognition model to identify the fluency of the text segment through the fluency recognition model.

[0186] The fluency recognition model is used to identify the fluency of text fragments. The features of the text fragments can be input into the fluency recognition model, which analyzes the features of the text fragments and outputs the fluency recognition result of the text fragments.

[0187] The fluency recognition model can be based on unsupervised training or supervised training. Fluency recognition models can be BERT models, convolutional neural network models, recurrent neural network models, etc.

[0188] Step S810: The feature vector of each character in the text segment is processed by the fluency recognition model to obtain the weight of each character in the text segment.

[0189] To improve the accuracy of fluency recognition results, the weight of each character in a text segment can be determined through a fluency recognition model. In other words, the feature vector of each character in the text segment can be input into the fluency recognition model, which can then process the feature vector of each character to obtain the weight of each character.

[0190] Optionally, the fluency recognition model can first convert the feature vector of each character in the text segment into a numerical value representing its importance, and then determine the weight of each character based on the numerical value corresponding to each character.

[0191] The fluency recognition model can convert the feature vector of each character in a text segment into a numerical value representing its importance using activation functions, including but not limited to the tanh function (hyperbolic tangent function) and the sigmoid function (S-shaped function). In one example, the numerical value corresponding to each character can be determined using the following formula:

[0192] S i = T tann(WV token_i +)

[0193] S i V represents the numerical value corresponding to the i-th character in the text segment, where i ranges from [1, w], w is the length of the text segment, and V token_i Let A be the feature vector of the i-th character included in the text segment, and let A, B, and W be parameter vectors. T As the transpose of A, A, B, and W can be adjusted during the training or optimization of the fluency recognition model.

[0194] After obtaining the numerical value corresponding to each character in the text segment, the fluency recognition model can normalize the numerical value of each character to obtain the weight of each character, thereby improving the accuracy of fluency recognition. In one example, the softmax function can be used to normalize the numerical value of each character, and the weight of each character can be determined by the following formula:

[0195]

[0196] It should be noted that, Figure 8 The specific implementation details of steps S310-S320 shown can be found in [reference]. Figure 3 Steps S310-S320 are shown. Figure 8 For detailed implementation of steps S710 and S730, please refer to [link / reference]. Figure 7 Steps S710 and S730 shown will not be repeated here.

[0197] exist Figure 8 In the illustrated embodiment, the feature vector of each character in the text segment is processed by the fluency recognition model to obtain the weight of each character in the text segment, which can improve the accuracy of fluency recognition of subsequent text segments.

[0198] In one exemplary embodiment, see Figure 9 , Figure 9 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0199] like Figure 9 As shown, the text processing method includes steps S310-S320, steps S910-S920, and step S340. The detailed descriptions of steps S910-S920 are as follows:

[0200] Step S910: Obtain a feature vector subsequence with a length matching the specified length from the feature vector sequence.

[0201] The specified length is used to characterize the number of feature vectors included in the feature vector subsequence. Since the feature vector subsequence corresponds to a text segment, the specified length can also characterize the number of characters included in the text segment. The specific value of the specified length can be flexibly set according to actual needs, for example, it can be 2, 3, 4, 5, 6, etc.

[0202] To obtain the features of a text segment in the text to be identified, multiple adjacent feature vectors can be obtained from the feature vector sequence to obtain a feature vector subsequence. The length of the feature vector subsequence matches a specified length, each feature vector corresponds to a character, and multiple adjacent feature vectors correspond to a text segment.

[0203] Optionally, to improve fluency recognition accuracy and avoid missed recognition, a feature vector subsequence of any length matching the specified length can be obtained from the feature vector sequence. For example, assuming the feature vector sequence is "c1, c2, c3, c4, c5, c6" and the specified length is 3, then the following four feature vector subsequences can be obtained: "c1, c2, c3", "c2, c3, c4", "c3, c4, c5", and "c4, c5, c6".

[0204] Alternatively, to save computational resources, multiple feature vector subsequences with a length matching the specified length can be obtained from the feature vector sequence, where there are no overlapping feature vectors between different feature vector subsequences; for example, assuming the feature vector sequence is "c1, c2, c3, c4, c5, c6" and the specified length is 3, then the two feature vector sets "c1, c2, c3" and "c4, c5, c6" can be obtained.

[0205] Step S920: Generate features of the text segment corresponding to the feature vector subsequence based on the feature vectors included in the feature vector subsequence.

[0206] Each feature vector subsequence corresponds to a text segment. After obtaining the feature vector subsequence from the feature vector sequence, the features of the corresponding text segment can be generated based on the feature vectors included in the feature vector subsequence. For example, suppose the feature vector sequence of "Today's weather is good" is: c1, c2, c3, c4, c5; with a specified number of 3, and "c1, c2, c3" is obtained from the feature vector sequence, then the features of the text segment "Today's weather" can be determined based on "c1, c2, c3".

[0207] The specific method for generating the features of the text segment corresponding to the feature vector subsequence based on the feature vectors included in the feature vector subsequence can be flexibly set according to actual needs. Optionally, it can be done through... Figure 7 Steps S720-S730 show the determination of the features of the text segment. That is, firstly, the weight of each feature vector in the feature vector subsequence is determined, and then, multiple feature vectors in the feature vector subsequence are fused according to the weights to obtain the features of the text segment corresponding to the feature vector subsequence.

[0208] It should be noted that, Figure 9 The specific implementation details of steps S310-S320 and step S340 shown can be found in [reference]. Figure 3 Steps S310-S320 and S340 shown will not be repeated here.

[0209] exist Figure 9 In the illustrated embodiment, a feature vector subsequence with a length matching a specified length is obtained from the feature vector sequence. Based on the feature vectors included in the feature vector subsequence, the features of the text segment corresponding to the feature vector subsequence are generated, which can improve the accuracy of subsequent text segment fluency recognition.

[0210] In one exemplary embodiment, see Figure 10 , Figure 10 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0211] like Figure 10 As shown, the text processing method includes steps S310-S320, S1010, S1020, S920, and S340. The detailed descriptions of steps S1010 and S1020 are as follows:

[0212] Step S1010: Obtain feature vector subsequences from the feature vector sequence using various strategies.

[0213] To improve the accuracy of text fragment fluency recognition and reduce the probability of missed recognition, different strategies can be set to obtain feature vector subsequences from the feature vector sequence.

[0214] Step S1020: In the process of obtaining a feature vector subsequence from the feature vector sequence using any strategy, a specified length corresponding to any strategy is obtained. Starting from each feature vector included in the feature vector sequence, a feature vector subsequence with a length matching the specified length corresponding to any strategy is obtained from the feature vector sequence according to a set direction. The specified lengths corresponding to different strategies are different.

[0215] To obtain feature vector subsequences of different lengths, different strategies require different specified lengths. For example, assuming there are three strategies with specified lengths of 3, 4, and 5 respectively, then a feature vector subsequence of length 3 can be obtained from the feature vector sequence; a feature vector subsequence of length 4 can be obtained from the feature vector sequence; and a feature vector subsequence of length 5 can be obtained from the feature vector sequence.

[0216] The direction can be set from front to back (i.e., the same direction as the reading order of the text to be recognized) or from back to front (i.e., the opposite direction to the reading order of the text to be recognized).

[0217] In the process of obtaining feature vector subsequences from a feature vector sequence using any strategy, the specified length corresponding to any strategy is first obtained. Then, taking each feature vector included in the feature vector sequence as the starting point, feature vector subsequences with lengths matching the specified lengths corresponding to the strategy are obtained from the feature vector sequence according to the set direction, thereby obtaining multiple feature vector subsequences corresponding to the strategy.

[0218] It should be noted that, Figure 10 The specific implementation details of steps S310-S320 and step S340 shown can be found in [reference]. Figure 3 Steps S310-S320 and step S340 are shown. Figure 10 For specific implementation details of step S920 shown, please refer to Figure 9 Step S920 shown will not be repeated here.

[0219] exist Figure 10In the illustrated embodiment, feature vector subsequences are obtained from the feature vector sequence using multiple strategies. Specifically, during the process of obtaining feature vector subsequences from the feature vector sequence using any one strategy, a specified length corresponding to any strategy is obtained. Starting from each feature vector included in the feature vector sequence, feature vector subsequences with lengths matching the specified length corresponding to any one strategy are obtained from the feature vector sequence according to a set direction. The specified lengths corresponding to different strategies are different, thereby enabling fluency recognition of text segments of different lengths in the text to be recognized, reducing the probability of missed recognition.

[0220] In one exemplary embodiment, see Figure 11 , Figure 11 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0221] like Figure 11 As shown, the text processing method includes steps S1110, S320-S330, and S1120. The detailed descriptions of steps S1110 and S1120 are as follows:

[0222] Step S1110: Extract the feature vector of each character in the text to be identified based on the feature extraction model obtained from unsupervised training.

[0223] To improve the generalization ability of text fluency recognition, the feature extraction model can be a model based on unsupervised training, that is, a model trained on training samples without labeled information; the feature extraction model extracts the context features of each character in the text to be recognized, and obtains the feature vector of each character.

[0224] Optionally, the feature extraction model can be a language model. Language models include, but are not limited to, statistical language models and neural language models. Language models can be one-way language models or two-way language models.

[0225] Step S1120: The features of the text segment are processed based on the fluency recognition model obtained by supervised training to obtain the fluency of the text segment.

[0226] To improve the accuracy of text fluency recognition, the fluency recognition model can be a model based on supervised training, that is, a model trained on training samples with labeled information.

[0227] To improve the fusion between the unsupervised training feature extraction model and the supervised training fluency recognition model, and thus improve the accuracy of fluency recognition, in one example, the context features of each character can be extracted based on the feature extraction model. Then, the context features and encoding features of each character are merged to obtain the merged features of each character. The merged feature sequence corresponding to the text to be recognized is input into the fluency recognition model, thereby generating the features of the text fragment based on the fluency recognition model.

[0228] It should be noted that, Figure 11 The specific implementation details of steps S320-S330 shown can be found in [reference]. Figure 3 Steps S320-S330 shown are not repeated here.

[0229] In related technologies, text processing methods mainly include text fluency recognition methods based on language models and text fluency recognition methods based on sequence labeling models. Language model-based text fluency recognition methods have good generalization ability but low accuracy. Sequence labeling model-based methods have higher accuracy but lower generalization ability. Furthermore, sequence labeling model-based methods typically determine text fluency based on whether a single character is erroneous; however, incoherence is usually caused by errors in a segment, and relying on single-character errors leads to a higher probability of missed recognitions. Figure 11 In the illustrated embodiment, feature vectors for each character are first extracted based on a feature extraction model obtained through unsupervised training. Then, based on the feature vectors of each character, a sequence of feature vectors corresponding to the text to be identified is generated. Features of the text segments included in the text to be identified are then generated based on the feature vector sequence. Finally, the fluency of the text segments is identified based on the features of the text segments. In this way, features from a model obtained through supervised training are deeply integrated with those from a model obtained through unsupervised training, achieving end-to-end learning. This allows the model to leverage both the strong generalization ability of the unsupervised training model and the high accuracy of the supervised training model, thereby improving the text fluency recognition effect. Furthermore, identifying fluency based on text segments reduces the probability of missed recognitions and makes it easier for the model to learn, improving the convergence speed of the model during training.

[0230] In one exemplary embodiment, see Figure 12 , Figure 12 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0231] like Figure 12 As shown, the text processing method includes steps S1110, S320-S330, S1120, and S1210-S1240. The detailed descriptions of steps S1210-S1240 are as follows:

[0232] Step S1210: Obtain the sample text and its annotation information, and extract the contextual features of each character in the sample text using a feature extraction model.

[0233] The sample text refers to the text used as a sample. The sample text has annotation information set to indicate whether there are any errors in the words in the sample text.

[0234] The feature extraction model can extract the contextual features of each character in the sample text. The specific method by which the feature extraction model extracts the contextual features of each character in the sample text is similar to the method by which the feature extraction model extracts the contextual features of each character in the text to be identified, and will not be elaborated here.

[0235] Step S1220: Generate features of sample text fragments included in the sample text based on the context features of each character included in the sample text; the sample text fragments include multiple characters.

[0236] A sample text fragment refers to a text fragment included in a sample text.

[0237] After obtaining the context features of each character in the sample text, features of the sample text fragments can be generated based on the context features of each character. Optionally, feature vectors for each character in the sample text can be generated based on the context features, and a sequence of feature vectors corresponding to the sample text can be generated based on the feature vectors of each character. Features of the sample text fragments can then be generated based on this sequence of feature vectors. The specific method for generating the features of the sample text fragments is similar to the method for generating the features of the text fragments included in the text to be identified, and will not be elaborated further here.

[0238] Step S1230: Analyze the features of the sample text fragment based on the fluency recognition model to obtain the fluency recognition result of the sample text fragment.

[0239] The features of the sample text fragments are input into the fluency recognition model to obtain the fluency recognition results of the sample text fragments.

[0240] Step S1240: Adjust the model parameters of the fluency recognition model based on the fluency recognition results and annotation information of the sample text fragments.

[0241] Based on the fluency recognition results of the sample text fragments included in the sample text and the annotation information of the sample text, the loss of the fluency recognition model can be determined. Based on the loss of the fluency recognition model, the model parameters of the fluency recognition model can be adjusted, thereby optimizing the fluency recognition model.

[0242] It should be noted that steps S1210-S1240 can be applied to the training phase of the fluency recognition model, or to the optimization phase after the fluency recognition model has been trained.

[0243] In addition, Figure 12 In the illustrated embodiment, steps S1110 and S1120 are executed first, followed by steps S1210-S1240. In practical applications, the model parameters of the fluency recognition model can be optimized based on steps S1210-S1240 first, and then the text to be recognized can be recognized based on steps S1110 and S1120.

[0244] It should be noted that, Figure 12 The specific implementation details of steps S320-S330 shown can be found in [reference]. Figure 3 Steps S320-S330 are shown. Figure 12 For detailed implementation of steps S1110 and S1120, please refer to [link / reference needed]. Figure 11 Steps S1110 and S1120 shown will not be repeated here.

[0245] exist Figure 12 In the illustrated embodiment, sample text and its annotation information are obtained, and the context features of each character in the sample text are extracted using a feature extraction model. Based on the context features of each character in the sample text, features of the sample text fragments included in the sample text are generated. The sample text fragments include multiple characters. The features of the sample text fragments are analyzed based on the fluency recognition model to obtain the fluency recognition results of the sample text fragments. Based on the fluency recognition results of the sample text fragments and the annotation information, the model parameters of the fluency recognition model are adjusted, so that the model can learn better and improve the convergence speed of the model during the training phase.

[0246] In one exemplary embodiment, see Figure 13 , Figure 13This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0247] like Figure 13 As shown, the text processing method includes steps S1110, S320-S330, S1120, S1210-S1230, and S1310-S1320. The detailed descriptions of steps S1310-S1320 are as follows:

[0248] Step S1310: Determine the number of annotated characters included in the sample text fragment based on the annotation information.

[0249] The annotation information is used to characterize the erroneous words in the sample text. Based on the annotation information, the number of annotated words included in the sample text fragment can be determined, that is, the number of erroneous words in the sample text fragment.

[0250] Step S1320: Adjust the model parameters of the fluency recognition model based on the number of annotated characters and the fluency recognition results of the sample text fragments.

[0251] The more characters that are annotated in a sample text segment, the higher the degree of incoherence of the sample text segment. Therefore, the loss of the fluency recognition model can be determined based on the number of annotated characters and the fluency recognition result of the sample text segment, thereby adjusting the model parameters of the fluency recognition model.

[0252] It should be noted that, Figure 13 The specific implementation details of steps S320-S330 shown can be found in [reference]. Figure 3 Steps S320-S330 are shown. Figure 13 For detailed implementation of steps S1110 and S1120, please refer to [link / reference needed]. Figure 11 Steps S1110 and S1120 are shown. Figure 13 For detailed implementation of steps S1210-S1230, please refer to [link / reference needed]. Figure 12 Steps S1210-S1230 shown are not repeated here.

[0253] exist Figure 13In the illustrated embodiment, the number of annotated characters in the sample text fragment is determined based on the annotation information. Based on the number of annotated characters and the fluency recognition result of the sample text fragment, the model parameters of the fluency recognition model are adjusted, thereby improving the model optimization effect.

[0254] In one exemplary embodiment, see Figure 14 , Figure 14 This is a flowchart illustrating a text processing method in another exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 110 in the implementation environment shown can execute this, or it can be performed by... Figure 1 The server 120 in the implementation environment shown can perform the operation, or it can be performed by... Figure 1 The terminal device 110 and server 120 in the implementation environment shown execute together.

[0255] like Figure 14 As shown, the text processing method includes steps S1110, S320-S330, S1120, S1210-S1230, S1310, and S1410-S1440. The detailed descriptions of steps S1410-S1440 are as follows:

[0256] Step S1410: Determine the ratio between the number of annotated characters and the length of the sample text segment.

[0257] The more characters that are marked in a sample text segment, the higher the degree of incoherence of the sample text segment. The ratio between the number of marked characters and the length of the sample text segment is a better indicator of the degree of incoherence. Therefore, after determining the number of marked characters in a sample text segment, the ratio between the number of marked characters and the length of the sample text segment can be determined.

[0258] In one example, the ratio between the number of annotated words in a sample text fragment and the length of the sample text fragment can be calculated using the following formula:

[0259] P label =N 标 / N 长

[0260] Where, N 标 N represents the number of annotated characters in the sample text segment. 长 P represents the length of the sample text fragment, i.e., the total number of words in the sample text fragment. label This represents the ratio corresponding to the sample text fragment.

[0261] For example, suppose the sample text is "Today's weather step number, it's raining and the wind is blowing". In this case, "step", "number" and "wind" are the characters that are incorrectly labeled. If the sample text fragment is "weather step number", then the number of labeled characters in the sample text fragment is 2, the length of the sample text fragment is 4, and the ratio of the sample text fragment is: 2 / 4 = 0.5.

[0262] Step S1420: Based on the fluency recognition results of the sample text fragments, generate a probability value representing the fluency of the sample text fragments.

[0263] After obtaining the fluency recognition results of the sample text fragments, a probability value representing the fluency of the sample text fragments can be generated based on the fluency recognition results.

[0264] The probability value representing the fluency of the sample text fragment can be flexibly set according to actual needs. In one example, the fluency recognition result of the sample text fragment can be converted into a probability value representing the fluency of the sample text fragment using the sigmoid function. The formula for calculating the probability value is as follows:

[0265] P pred =sigmoid(QV w )

[0266] Among them, QV w P represents the fluency recognition result of the sample text fragment. pred This represents the probability value corresponding to the sample text fragment.

[0267] Step S1430: Calculate the loss of the fluency recognition model based on the difference between the probability value and the ratio.

[0268] The loss of the fluency recognition model can be calculated based on the difference between the ratio and the probability value.

[0269] The method for calculating the difference between the ratio and the probability value can be flexibly set according to actual needs. In one example, the difference between the probability value and the ratio can be determined based on the cross-entropy function. For instance, the difference between the ratio and the probability value can be calculated using the following formula:

[0270] loss = -P label logP pred -(1-P label log(1-P) pred )

[0271] Here, loss is the difference between the ratio and probability value corresponding to the sample text segment.

[0272] If the number of sample text segments is 1, the difference value corresponding to that sample text segment can be directly used as the loss of the fluency recognition model. If the number of sample text segments is multiple, the loss of the fluency recognition model can be determined based on the difference values ​​corresponding to multiple text segments. In one example, the loss of the fluency recognition model can be calculated using the following formula:

[0273]

[0274] Where, loss j Let be the difference value corresponding to the j-th sample text segment, n be the number of text segments, and Loss be the loss of the fluency recognition model.

[0275] Step S1440: Adjust the model parameters of the fluency recognition model based on the loss of the fluency recognition model.

[0276] After calculating the loss of the fluency recognition model, the model parameters of the fluency recognition model are adjusted based on the loss.

[0277] It should be noted that, Figure 14 The specific implementation details of steps S320-S330 shown can be found in [reference]. Figure 3 Steps S320-S330 are shown. Figure 14 For detailed implementation of steps S1110 and S1120, please refer to [link / reference needed]. Figure 11 Steps S1110 and S1120 are shown. Figure 14 For detailed implementation of steps S1210-S1230, please refer to [link / reference needed]. Figure 12 Steps S1210-S1230 are shown. Figure 14 For detailed implementation of step S1310 shown, please refer to [link / reference]. Figure 13 Step S1310 shown will not be repeated here.

[0278] exist Figure 14 In the illustrated embodiment, the ratio between the number of annotated characters and the length of the sample text segment is determined. Based on the fluency recognition result of the sample text segment, a probability value representing the fluency of the sample text segment is generated. Based on the difference between the probability value and the ratio, the loss of the fluency recognition model is calculated. The model parameters of the fluency recognition model are then adjusted according to the loss of the fluency recognition model, which can improve the model optimization effect and enable the model to learn the logical relationship between the features and fluency of the text segment more quickly.

[0279] In one exemplary embodiment, see Figure 15 , Figure 15This is another exemplary embodiment of the present application illustrating the overall architecture of a fluency recognition model that integrates a converter-based bidirectional language model and a BERT sequence labeling model. Figure 15 As shown, the fluency recognition model includes a transducer-based bidirectional language model and a BERT sequence labeling model. See also Figure 16 As shown, Figure 16 This is another exemplary embodiment of the present application illustrating the basis. Figure 15 The flowchart illustrates the text processing method of the fluency recognition model. Figure 16 As shown, the text processing method includes steps S1601-S1606, which are described in detail below:

[0280] Step S1601: Input the text to be recognized into the bidirectional language model based on the converter to obtain the representation vector of each character in the text to be recognized, which is output by the bidirectional language model based on the converter; wherein, the representation vector of each character includes the context features of the character.

[0281] In this embodiment of the application, the text to be recognized can be concatenated into a character sequence according to the following rules and then input into a bidirectional language model based on a converter:

[0282] [CLS]Char1 Char2 Char3 Char4……CharL[SEP]

[0283] Here, [CLS] and [SEP] are predefined markers used to indicate the beginning and end of the text. Char1, Char2, Char3, Char4, and CharL represent the characters appearing sequentially in the text to be recognized, and L is the total number of characters in the text. For example, see... Figure 15 As shown, if the text to be recognized is "Flying a plane is both physical and mental work", then the text to be recognized is converted into the character sequence "[CLS]Flying a plane is both physical and mental work [SEP]", and then input into the bidirectional language model based on the converter.

[0284] Since unidirectional neural language models can only utilize one-sided context, this embodiment uses a converter-based bidirectional language model to determine the representation vector of each character in the text to be recognized. The representation vector of each character includes the contextual features of that character. See also... Figure 17 As shown, Figure 17Schematic diagram of the composition architecture of a Transformer-based bidirectional language model provided by an embodiment of this application. The Transformer-based bidirectional language model includes a unidirectional language model that predicts from front to back and a unidirectional language model that predicts from back to front. That is, the unidirectional language model that predicts from front to back can extract the upstream features of each character, and the unidirectional language model that predicts from back to front can extract the downstream features of each character. In the process of extracting the context features of each character in the text to be recognized, the unidirectional language model that predicts from front to back can determine the representation vector of the previous character of each character in the process of predicting from front to back, and use the determined representation vector as the upstream feature of the character; the unidirectional language model that predicts from back to front can determine the representation vector of the next character of each character in the process of predicting from back to front, and use the determined representation vector as the downstream feature of the character; then, the upstream feature and the downstream feature of each character are concatenated to obtain the representation vector of each character. For example, assuming that the text to be recognized is "Let citizens have a sense of gain and happiness", for the character "Xin", the representation vectors of the previous character "Gan" and the next character "Fu" are concatenated to obtain the representation vector of "Xin".

[0285] The Transformer-based bidirectional language model provided by an embodiment of this application can be obtained through unsupervised training, that is, trained based on training samples without annotation information. Therefore, it has strong generalization ability. When training the Transformer-based bidirectional language model, the two unidirectional language models can be trained together. In this way, the context in both directions can be utilized simultaneously during the training process, and only one pass of model prediction on the text is required to obtain the context features of each character included in the text, improving the model training speed and prediction speed.

[0286] Step S1602: Encoding each character included in the text to be recognized to obtain the original BERT word vector of each character.

[0287] Among them, any vectorization encoding method can be used to encode each character in the text to be recognized to obtain the original BERT word vector of each character.

[0288] Step S1603: Merging the representation vector and the original BERT word vector of each character included in the text to be recognized to obtain a merged feature vector.

[0289] As Figure 15 shown, adding the representation vector and the original BERT word vector of each character to obtain the merged feature vector of the character.

[0290] Step S1604: Inputting the merged feature vector of each character included in the text to be recognized into the BERT sequence labeling model.

[0291] The BERT sequence labeling model can be obtained through supervised training; its training method can be found in [reference needed]. Figures 12-14 The details of that record will not be repeated here.

[0292] like Figure 15 As shown, the representation vector of each character is added to the original BERT character vector, and then the merged feature vector obtained by the addition is input into the BERT sequence labeling model.

[0293] Step S1605: Process the merged feature vector of each character using the BERT sequence labeling model to obtain the features of the text segments included in the text to be identified.

[0294] like Figure 15 As shown, the BERT sequence labeling model processes the merged feature vector of each character to obtain the features of the text segments included in the text to be identified.

[0295] Optionally, the BERT sequence labeling model can first process the merged feature vector sequence corresponding to the text to be identified to obtain the feature vector of each character included in the text to be identified, and then obtain the feature vector sequence corresponding to the text to be identified.

[0296] After obtaining the feature vector sequence, feature vector subsequences can be extracted from it using windows of different lengths. Each window, after extracting one feature vector subsequence, moves forward one feature vector and extracts the next, until the window reaches the last feature vector in the feature vector sequence. For example, see... Figure 18 As shown, assuming the character sequence "[CLS] Flying a plane consumes physical strength [SEP]", its corresponding feature vector sequence is "g1 g2g3 g4 g5 g6 g7 g8", including windows of length 3, 4, and 5. For a window of length 3, starting from g1, after extracting "g1 g2 g3", move one feature vector forward and extract "g2 g3 g4", and so on, until "g6 g7 g8" is extracted. Similarly, for a window of length 4, starting from g1, after extracting "g1 g2 g3 g4", move one feature vector forward and extract "g2 g3 g4 g5", and so on, until "g5 g6 g7 g8" is extracted. For a window of length 5, starting from g1, after extracting "g1 g2 g3 g4 g5", move one feature vector forward and extract "g2 g3 g4 g5". "g6", and so on, until the window extracts "g4 g5 g6 g7 g8".

[0297] After extracting multiple subsequences of feature vectors with different lengths, for each subsequence of feature vectors, through the self-attention layer of the BERT sequence labeling model, the weight of each feature vector included in each subsequence of feature vectors is determined, and the feature vectors included in each subsequence of feature vectors are fused according to the weights to obtain the features of the text segment corresponding to each subsequence of feature vectors. For example, by fusing "g1 g2 g3", the features of the text segment "[CLS] Take off" are obtained.

[0298] Step S1606, identify the features of the text segment through the BERT sequence labeling model to obtain the recognition result of the smoothness of the text segment.

[0299] As Figure 15 shown, after generating the features of the text segment, the smoothness of the text segment can be identified through the BERT sequence labeling model.

[0300] It can be understood that Figure 16 the specific implementation details of each step in the illustrated embodiment have been introduced in detail in the foregoing embodiments and will not be elaborated herein.

[0301] Figure 19 is a block diagram of a text processing device shown in an embodiment of the present application. As Figure 19 shown, the device includes:

[0302] Extraction module 1901, configured to extract the feature vector of each character included in the text to be recognized, and the feature vector of each character includes the context feature of each character;

[0303] Sequence generation module 1902, configured to generate a sequence of feature vectors corresponding to the text to be recognized according to the feature vectors of each character;

[0304] Feature generation module 1903, configured to generate the features of the text segments included in the text to be recognized according to the sequence of feature vectors, and the text segments include multiple characters;

[0305] Recognition module 1904, configured to recognize the smoothness of the text segment according to the features of the text segment.

[0306] In an embodiment of the present application, the extraction module 1901 includes:

[0307] Context feature extraction module, configured to extract the context feature of each character included in the text to be recognized;

[0308] Encoding module, configured to encode each character to obtain the encoded feature of each character;

[0309] Extraction sub-module, configured to generate the feature vector of each character according to the context feature and the encoded feature of each character.

[0310] In one embodiment of this application, the context feature extraction module includes:

[0311] The context feature and context feature generation module is configured to generate context features for each character based on the relationship between each character and its context in the text to be recognized, and to generate context features for each character based on the relationship between each character and its context in the text to be recognized.

[0312] The context feature determination module is configured to determine the context features of each character based on the preceding and following context features.

[0313] In one embodiment of this application, the extraction submodule is specifically configured as follows:

[0314] The context features and encoding features of each character are merged to obtain the merged features of each character;

[0315] Based on the merging features of each character, a merging feature sequence corresponding to the text to be recognized is generated. The merging feature sequence includes the merging features of each character.

[0316] The association between each character and other characters in the text to be identified is calculated based on the merged feature sequence.

[0317] Each character's feature vector is generated based on its relationship with other characters.

[0318] In one embodiment of this application, the feature generation module 1903 includes:

[0319] The vector acquisition module is configured to acquire a subsequence of feature vectors with a length matching a specified length from the feature vector sequence;

[0320] The feature generation submodule is configured to generate features of the text fragments corresponding to the feature vector subsequence based on the feature vectors included in the feature vector subsequence.

[0321] In one embodiment of this application, the vector acquisition module is specifically configured as follows:

[0322] Multiple strategies are used to obtain feature vector subsequences from feature vector sequences;

[0323] In the process of obtaining a feature vector subsequence from a feature vector sequence using any strategy, a specified length corresponding to any strategy is obtained. Starting from each feature vector included in the feature vector sequence, a feature vector subsequence with a length matching the specified length corresponding to any strategy is obtained from the feature vector sequence according to a set direction. The specified lengths are different for different strategies.

[0324] In one embodiment of this application, the recognition module 1904 is specifically configured to: input the features of the text segment into a pre-trained fluency recognition model, so as to recognize the fluency of the text segment through the fluency recognition model;

[0325] The feature generation module is specifically configured as follows: obtaining the feature vector of each character included in the text segment from the feature vector sequence; processing the feature vector of each character included in the text segment through the fluency recognition model to obtain the weight of each character included in the text segment; and fusing the feature vectors of multiple characters included in the text segment according to the obtained weights to obtain the features of the text segment.

[0326] In one embodiment of this application, the extraction module 1901 is specifically configured to: extract the feature vector of each character included in the text to be identified based on the feature extraction model obtained by unsupervised training;

[0327] The recognition module 1904 is specifically configured to process the features of text segments based on a fluency recognition model obtained through supervised training in order to obtain the fluency of the text segments.

[0328] In one embodiment of this application, the device further includes:

[0329] The sample feature extraction module is configured to acquire sample text and its annotation information, and extract the contextual features of each character in the sample text using a feature extraction model.

[0330] The sample text fragment feature generation module is configured to generate features of the sample text fragments included in the sample text based on the context features of each character included in the sample text; the sample text fragments include multiple characters.

[0331] The sample recognition module is configured to analyze the features of sample text fragments based on a fluency recognition model to obtain the fluency recognition results of the sample text fragments.

[0332] The adjustment module is configured to adjust the model parameters of the fluency recognition model based on the fluency recognition results and annotation information of the sample text fragments.

[0333] In one embodiment of this application, the adjustment module includes:

[0334] The quantity determination module is configured to determine the number of annotated characters included in the sample text fragment based on the annotation information;

[0335] Adjust the submodule to adjust the model parameters of the fluency recognition model based on the number of annotated characters and the fluency recognition results of the sample text fragments.

[0336] In one embodiment of this application, the adjustment submodule is specifically configured as follows:

[0337] Determine the ratio between the number of annotated words and the length of the sample text segment;

[0338] Based on the fluency recognition results of the sample text fragments, a probability value representing the fluency of the sample text fragments is generated;

[0339] The loss of the fluency recognition model is calculated based on the difference between the probability value and the ratio.

[0340] Adjust the model parameters of the fluency recognition model based on the loss of the fluency recognition model.

[0341] It should be noted that the apparatus provided in the foregoing embodiments and the method provided in the foregoing embodiments belong to the same concept, and the specific way in which each module and unit performs operations has been described in detail in the method embodiments.

[0342] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more computer programs, which, when executed by one or more processors, cause the electronic device to perform the aforementioned text processing method.

[0343] Figure 20 This is a schematic diagram of the structure of a computer system suitable for implementing the electronic devices of the present application embodiments.

[0344] It should be noted that, Figure 20 The computer system 2000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0345] like Figure 20 As shown, the computer system 2000 includes a Central Processing Unit (CPU) 2001, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 2002 or programs loaded from Storage Unit 2008 into Random Access Memory (RAM) 2003. The RAM 2003 also stores various programs and data required for system operation. The CPU 2001, ROM 2002, and RAM 2003 are interconnected via a bus 2004. An Input / Output (I / O) interface 2005 is also connected to the bus 2004.

[0346] The following components are connected to I / O interface 2005: an input section 2006 including a keyboard, mouse, etc.; an output section 2007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 2008 including a hard disk, etc.; and a communication section 2009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 2009 performs communication processing via a network such as the Internet. Drive 2010 is also connected to I / O interface 2005 as needed. Removable media 2011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 2010 as needed so that computer programs read from them can be installed into storage section 2008 as needed.

[0347] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 2009, and / or installed from removable medium 2011. When the computer program is executed by central processing unit (CPU) 2001, it performs various functions defined in the system of this application.

[0348] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. For example, a computer-readable medium can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium that includes or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program included on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0349] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0350] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0351] Another aspect of this application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned text processing method. This computer-readable medium may be included in the electronic device described in the above embodiments, or it may exist independently without being assembled into the electronic device.

[0352] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable medium. A processor of a computer device reads the computer instructions from the computer-readable medium and executes the computer instructions, causing the computer device to perform the text processing methods provided in the various embodiments described above.

[0353] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A text processing method, characterized in that, The method includes: The feature extraction model, trained under unsupervised supervision, merges the contextual and encoding features of each character in the text to be recognized to obtain a merged feature for each character. Based on the merged feature of each character, a merged feature sequence corresponding to the text to be recognized is generated, wherein the merged feature sequence includes the merged feature of each character. The association between each character and other characters in the text to be recognized is calculated based on the merged feature sequence. Based on the association between each character and other characters, a feature vector of each character is generated to extract the feature vector of each character in the text to be recognized. Based on the feature vector of each character, generate a sequence of feature vectors corresponding to the text to be identified; The feature vectors of each character in the text segment of the text to be identified are obtained from the feature vector sequence. The feature vectors of each character in the text segment are then processed using a fluency recognition model obtained through supervised training to obtain an importance score for each character in the text segment. Based on the importance scores corresponding to the multiple characters in the text segment, the importance scores of each character in the text segment are normalized to obtain a weight for each character in the text segment. The feature vectors of the multiple characters in the text segment are then fused based on the obtained weights to obtain the features of the text segment, thereby generating the features of the text segment based on the feature vector sequence. Based on the characteristics of the text fragment, the fluency of the text fragment is identified.

2. The method as described in claim 1, characterized in that, The step of extracting the feature vector of each character in the text to be identified includes: Extract the contextual features of each character in the text to be identified; Each character is encoded to obtain the encoding features of each character; A feature vector for each character is generated based on the contextual and encoding features of each character.

3. The method as described in claim 2, characterized in that, The extraction of contextual features for each character in the text to be identified includes: Based on the relationship between each character and its preceding context in the text to be identified, a preceding context feature is generated for each character; and based on the relationship between each character and its following context in the text to be identified, a following context feature is generated for each character. The context features of each character are determined based on its preceding and following context features.

4. The method as described in claim 1, characterized in that, The features for generating the text segment based on the feature vector sequence include: Obtain a subsequence of feature vectors with a length matching the specified length from the feature vector sequence; Based on the feature vectors included in the feature vector subsequence, the features of the text segment corresponding to the feature vector subsequence are generated.

5. The method as described in claim 4, characterized in that, The step of obtaining a subsequence of feature vectors with a length matching a specified length from the feature vector sequence includes: Feature vector subsequences are obtained from the feature vector sequence using multiple strategies; In the process of obtaining a feature vector subsequence from the feature vector sequence using any strategy, a specified length corresponding to any strategy is obtained. Starting from each feature vector included in the feature vector sequence, a feature vector subsequence with a length matching the specified length corresponding to any strategy is obtained from the feature vector sequence according to a set direction. The specified lengths are different for different strategies.

6. The method as described in claim 1, characterized in that, The step of identifying the fluency of a text segment based on its features includes: inputting the features of the text segment into the fluency recognition model to identify the fluency of the text segment through the fluency recognition model.

7. The method as described in claim 1, characterized in that, The method further includes: Obtain sample text and its annotation information, and extract the contextual features of each character in the sample text using the feature extraction model; Based on the contextual features of each character in the sample text, features of the sample text fragments included in the sample text are generated; the sample text fragments include multiple characters. Based on the fluency recognition model, the features of the sample text fragment are analyzed to obtain the fluency recognition result of the sample text fragment; Based on the fluency recognition results of the sample text fragments and the annotation information, the model parameters of the fluency recognition model are adjusted.

8. The method as described in claim 7, characterized in that, The step of adjusting the model parameters of the fluency recognition model based on the fluency recognition result of the sample text fragment and the annotation information includes: The number of annotated characters included in the sample text fragment is determined based on the annotation information; The model parameters of the fluency recognition model are adjusted based on the number of labeled characters and the fluency recognition results of the sample text fragment.

9. The method as described in claim 8, characterized in that, The step of adjusting the model parameters of the fluency recognition model based on the number of labeled characters and the fluency recognition result of the sample text segment includes: Determine the ratio between the number of annotated characters and the length of the sample text segment; Based on the fluency recognition results of the sample text fragment, a probability value representing the fluency of the sample text fragment is generated; The loss of the fluency recognition model is calculated based on the difference between the probability value and the ratio. The model parameters of the fluency recognition model are adjusted based on the loss of the fluency recognition model.

10. A text processing device, characterized in that, The device includes: The extraction module is configured to merge the context features and encoding features of each character in the text to be recognized based on an unsupervised training feature extraction model, thereby obtaining the merged features of each character. Based on the merged features of each character, a merged feature sequence corresponding to the text to be recognized is generated, wherein the merged feature sequence includes the merged features of each character. The module then calculates the association between each character and other characters in the text to be recognized, based on the merged feature sequence. Finally, based on the association between each character and the other characters, a feature vector of each character is generated to extract the feature vector of each character in the text to be recognized. The sequence generation module is configured to generate a sequence of feature vectors corresponding to the text to be identified based on the feature vector of each character. The feature generation module is configured to obtain the feature vector of each character in the text segment of the text to be identified from the feature vector sequence, and process the feature vector of each character in the text segment using a fluency recognition model obtained through supervised training to obtain the importance score of each character in the text segment; normalize the importance score of each character in the text segment according to the importance scores corresponding to the multiple characters in the text segment to obtain the weight of each character in the text segment; and fuse the feature vectors of the multiple characters in the text segment according to the obtained weights to obtain the features of the text segment, so as to generate the features of the text segment based on the feature vector sequence. The recognition module is configured to identify the fluency of the text segment based on its features.

11. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more computer programs that, when executed by one or more processors, cause the electronic device to implement the text processing method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the processor of the electronic device, causes the electronic device to implement the text processing method according to any one of claims 1-9.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the text processing method according to any one of claims 1-9.

Images