Text fluency evaluation method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a text fluency evaluation method, device, electronic equipment and storage medium. The method comprises: obtaining text data to be evaluated for fluency; performing word segmentation processing on the text data, and generating a first text fluency evaluation plan based on the word segmentation result; obtaining a text fluency cache library; correcting the first text fluency evaluation plan according to the comparison result of the cache text and the text data to obtain a second text fluency evaluation plan; executing the second text fluency evaluation plan, and calculating the second text fluency corresponding to the text data according to the first text fluency and the matching fluency obtained by evaluating the second text fluency evaluation plan. The embodiment of the present disclosure can improve the text fluency evaluation efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for evaluating text fluency. Background Technology

[0002] When inputting text via a computer device, multiple candidate texts are generated based on the string entered by the user for the user to choose from. When the entered text has related preceding text, the candidate text recommended to the user is determined by evaluating the fluency between the preceding text and each piece of text. This improves the efficiency of text input for users.

[0003] In related technologies, large language models can be used to evaluate the text fluency between the associated preceding content and candidate texts, and multiple candidate texts can be ranked based on the fluency evaluation results, thereby improving the accuracy of candidate text recommendation. However, in related technologies, the efficiency of text fluency evaluation is relatively low. Summary of the Invention

[0004] This disclosure provides a text fluency assessment method, apparatus, electronic device, and storage medium that can improve the efficiency of text fluency assessment.

[0005] According to one aspect of this disclosure, a method for assessing text fluency is provided, comprising:

[0006] Obtain the text data to be evaluated for fluency, which consists of candidate text corresponding to the string input by the input object and related preceding text;

[0007] The text data is segmented into words, and a first text fluency assessment plan is generated based on the segmentation results;

[0008] Obtain a text fluency cache library, which contains multiple cached texts and a fluency evaluation value for each cached text;

[0009] The first text fluency evaluation plan is modified based on the comparison result between the cached text and the text data to obtain a second text fluency evaluation plan. The comparison result includes the matching text and the matching fluency corresponding to the matching text. The matching text is a part of the text in the text data.

[0010] The second text fluency assessment plan is executed, and the second text fluency corresponding to the text data is calculated based on the first text fluency obtained from the second text fluency assessment plan and the matching fluency.

[0011] According to one aspect of this disclosure, a text fluency assessment apparatus is provided, comprising:

[0012] The first acquisition unit is used to acquire text data to be evaluated for fluency, wherein the text data consists of candidate text corresponding to the string input by the input object and related preceding text;

[0013] The word segmentation unit is used to perform word segmentation processing on the text data and generate a first text fluency evaluation plan based on the word segmentation results;

[0014] The second acquisition unit is used to acquire a text fluency cache library, which contains multiple cached texts and a fluency evaluation value corresponding to each cached text.

[0015] The correction unit is used to correct the first text fluency evaluation plan based on the comparison result between the cached text and the text data to obtain a second text fluency evaluation plan. The comparison result includes the matching text and the matching fluency corresponding to the matching text. The matching text is a part of the text in the text data.

[0016] The calculation unit is configured to execute the second text fluency assessment plan and calculate the second text fluency corresponding to the text data based on the first text fluency assessed by executing the second text fluency assessment plan and the matching fluency.

[0017] Optionally, in one embodiment, the correction unit is specifically used for:

[0018] The text data is compared with the cached text in the text fluency cache library, and the target cached text is determined in the text fluency cache library based on the comparison result;

[0019] Based on the target cached text, at least one redundant step is identified among the multiple first evaluation steps included in the first text fluency evaluation plan;

[0020] The first text fluency assessment plan is modified according to the at least one redundant step to obtain the second text fluency assessment plan.

[0021] Optionally, in one embodiment, the correction unit is specifically used for:

[0022] The multiple words determined by the word segmentation result are sorted according to the text order in the text data, and multiple first continuous word groups containing the source word are generated based on the sorting order and the multiple words, wherein the source word is the word sorted first.

[0023] The plurality of first consecutive word groups are compared with the cached text in the text fluency cache library to obtain a plurality of matching second consecutive word groups;

[0024] The consecutive word group containing the most word segments among the plurality of second consecutive word groups is identified as the target cached text.

[0025] Optionally, in one embodiment, the correction unit is specifically used for:

[0026] Obtain the evaluation target corresponding to each first evaluation step in the first text fluency evaluation plan, wherein the evaluation target includes the evaluation text and the evaluation fluency value;

[0027] The first evaluation step that determines whether the evaluation text matches the target cached text is the matching step;

[0028] The preceding steps of the matching step are determined based on the execution order among the multiple first evaluation steps in the first text fluency evaluation plan, and at least one redundant step is determined based on the preceding steps and the matching steps.

[0029] Optionally, in one embodiment, the correction unit is specifically used for:

[0030] By removing the redundant steps from the first text fluency assessment plan, multiple second assessment steps are obtained.

[0031] The second text fluency assessment plan is generated based on the execution order of the multiple second assessment steps and the steps in the first text fluency assessment plan.

[0032] Optionally, in one embodiment, the text fluency cache library further includes model state parameters corresponding to each cached text, the model state parameters indicating the model state of the preset neural network model after calculating the fluency evaluation value corresponding to the cached text;

[0033] The computing unit is specifically used for:

[0034] Retrieve the matching model state parameters corresponding to the matching text from the text fluency cache library;

[0035] The preset neural network model is updated based on the matching model state parameters, and the second text fluency evaluation plan is executed using the updated preset neural network model.

[0036] Optionally, in one embodiment, the text fluency assessment device further includes:

[0037] The third acquisition unit is used to determine at least one text to be stored based on the text data and the matching text, and to acquire the text to be stored fluency evaluation value corresponding to each text to be stored based on the execution process of the second text fluency evaluation plan.

[0038] An update unit is used to update the text fluency cache based on the text to be stored and the corresponding fluency evaluation value to be stored.

[0039] Optionally, in one embodiment, the text fluency assessment device further includes:

[0040] The fourth acquisition unit is used to acquire the usage frequency and recent usage time of each cached text in the text fluency cache when the text fluency cache reaches a predetermined condition;

[0041] A filtering unit is used to filter multiple cached texts in the text fluency cache library based on the usage frequency and the most recent usage time.

[0042] Optionally, in one implementation, the filtering unit is specifically used for:

[0043] Obtain the effective text length for each cached text;

[0044] The multiple cached texts in the text fluency cache are filtered based on the usage frequency, the most recent usage time, and the effective text length.

[0045] Optionally, in one implementation, the filtering unit is specifically used for:

[0046] For each cached text target text content, query other cached texts for overlapping content with the target text content;

[0047] When the intersection content exists in the other cached text, the length of the target text content excluding the intersection content is determined as the effective text length;

[0048] If the intersection content does not exist in the other cached texts, the length of the target text content is determined as the effective text length.

[0049] Optionally, in one implementation, the filtering unit is specifically used for:

[0050] A first computational cost is calculated based on the frequency of use, a second computational cost is calculated based on the most recent usage time, and a third computational cost is calculated based on the effective text length.

[0051] The total computation cost of the corresponding cached text is determined based on the first computation cost, the second computation cost, and the third computation cost.

[0052] The multiple cached texts in the text fluency cache library are filtered based on the total computational cost corresponding to each cached text.

[0053] Optionally, in one embodiment, the second acquisition unit is specifically used for:

[0054] Obtain the text identifier corresponding to the text data;

[0055] Based on the text identifier, determine the text fluency cache library corresponding to the text data from multiple candidate text fluency cache libraries.

[0056] In the text fluency evaluation method of this embodiment, the following steps are taken: First, text data to be evaluated for fluency is obtained. The text data consists of candidate text corresponding to a string input by an input object and related preceding text. Second, the text data is segmented into words, and a first text fluency evaluation plan is generated based on the segmentation results. Third, a text fluency cache library is obtained, containing multiple cached texts and a fluency evaluation value corresponding to each cached text. Fourth, the first text fluency evaluation plan is modified according to the comparison results between the cached texts and the text data to obtain a second text fluency evaluation plan. The comparison results contain matching texts and their corresponding matching fluency values. The matching texts are a portion of the text data. Fifth, the second text fluency evaluation plan is executed, and the second text fluency corresponding to the text data is calculated based on the first text fluency and the matching fluency obtained from the second text fluency evaluation plan.

[0057] Therefore, in this embodiment, when performing text fluency evaluation, matching text that has already undergone fluency evaluation can be retrieved from the text fluency cache, and the text fluency evaluation plan can be revised based on the matching text. Thus, when executing the second text fluency evaluation plan, it is not necessary to start the fluency evaluation of the text data from the first word segment; instead, the fluency evaluation can be performed on the remaining text content excluding the matching text based on the matching text, and the second text fluency corresponding to the text data can be calculated using the matching fluency corresponding to the matching text and the first text fluency corresponding to the remaining text content. Therefore, this embodiment can fully reuse the fluency evaluation values ​​of some text cached in the text fluency cache, thereby greatly reducing the computational load when evaluating the fluency of text data and improving the efficiency of text fluency evaluation.

[0058] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0059] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0060] Figure 1 This is a system architecture diagram of the text fluency assessment method applied according to embodiments of the present disclosure;

[0061] Figure 2 This is a schematic diagram illustrating the application of an embodiment of this disclosure in a Chinese input scenario;

[0062] Figure 3 This is a flowchart of a text fluency assessment method according to an embodiment of the present disclosure;

[0063] Figure 4 This is a schematic diagram illustrating the calculation of a smoothness assessment value using a preset neural network model according to an embodiment of the present disclosure;

[0064] Figure 5 This is a schematic diagram of a text fluency caching library according to one embodiment of the present disclosure;

[0065] Figure 6 This is another schematic diagram of a text fluency cache library according to one embodiment of the present disclosure;

[0066] Figure 7 This is another schematic diagram of a text fluency cache library according to one embodiment of the present disclosure;

[0067] Figure 8 This is a schematic diagram illustrating the acquisition of a text fluency cache library based on a text identifier according to one embodiment of this disclosure;

[0068] Figure 9 This is a schematic diagram illustrating the determination of the fluency assessment value of the text to be stored according to one embodiment of the present disclosure;

[0069] Figure 10 This is a schematic diagram illustrating data changes in a text fluency cache library according to an embodiment of the present disclosure;

[0070] Figure 11 This is another schematic flowchart of a text fluency assessment method according to an embodiment of the present disclosure;

[0071] Figure 12 This is a schematic diagram of the structure of a text fluency assessment device according to an embodiment of the present disclosure;

[0072] Figure 13 This is a terminal structure diagram for implementing various methods according to an embodiment of the present disclosure;

[0073] Figure 14 This is a server structure diagram illustrating the implementation of various methods according to an embodiment of the present disclosure. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0075] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0076] Neural network models are mathematical models based on artificial neurons, used to simulate the neural network structure and function of the human brain. They are complex network systems formed by a large number of simple processing units (called neurons) extensively interconnected. Neural network models can be used to handle complex problems that only the human brain can process, such as semantic recognition and image recognition.

[0077] Bayesian probability is a probability calculation method based on statistical principles. It obtains the posterior probability by updating the prior probability, thus describing the probability of an event occurring given certain conditions. Bayes' theorem is the core of Bayesian probability; it describes how to calculate the posterior probability given the prior probability and the likelihood function. Bayes' theorem can be expressed as P(A|B), which represents the conditional probability (posterior probability) of event A occurring given that event B has occurred.

[0078] A Trie Tree (TBT) is a tree-structured data structure used for quickly retrieving keys from a string dataset. Its core idea is to store the prefixes of each string as paths in the tree, thus enabling efficient searching and matching.

[0079] In related technologies, when evaluating the fluency of a text using a large language model, a pre-established fluency cache can be used to check whether the text's fluency has already been evaluated. If the fluency cache contains the text's fluency result, it can be directly obtained; however, if the cache does not contain the result, fluency evaluation needs to be performed word by word from the beginning of the text segmentation. Thus, when the cache does not contain the fluency evaluation result, the efficiency of the fluency evaluation is low. Therefore, this disclosure provides a text fluency evaluation method to improve its efficiency.

[0080] System architecture and scenario description of the embodiments disclosed herein

[0081] Figure 1 This is a system architecture diagram applied to the text fluency evaluation method according to embodiments of the present disclosure. It includes a terminal 140, an Internet 130, a gateway 120, a server 110, etc.

[0082] Terminal 140 can take various forms, including desktop computers, laptops, PDAs (personal digital assistants), mobile phones, in-vehicle terminals, home theater terminals, and dedicated terminals. Furthermore, it can be a single device or a collection of multiple devices. For example, multiple devices can be connected via a local area network, sharing a single display device to work collaboratively, forming a single terminal 140. Terminal 140 can also communicate with the Internet 130 via wired or wireless means to exchange data.

[0083] Server 110 refers to a computer system that can provide certain services to terminal 140. Compared to ordinary terminal 140, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a machine), or a combination of portions of multiple high-performance computers (e.g., machines).

[0084] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator between two systems using different communication protocols, data formats, languages, or even completely different architectures. It enables network interconnection at the transport layer. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 through gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 through gateway 120.

[0085] The text fluency assessment method of this disclosure embodiment can be executed by an electronic device, which can be a terminal 140 or a server 110. That is, the text fluency assessment method can be executed by the terminal 140 or the server 110, or it can be executed by both the terminal 140 and the server 110.

[0086] The embodiments disclosed herein can be applied in various scenarios, such as Figure 2 The examples shown include Chinese input scenarios.

[0087] The text input interface 210 includes a text input box 211. Objects can enter Chinese text content into the text input box 211. For example... Figure 2As shown, the user has entered "Welcome new" in the text input box 211, and then entered the pinyin "pengyou". After receiving the pinyin string entered by the user, the electronic device implementing the present disclosure can obtain multiple candidate words and calculate the smoothness between each candidate word and the above content.

[0088] The text smoothness evaluation method of the embodiments of the present disclosure can be used to evaluate the smoothness score of the text content composed of the above content and each candidate word. The higher the smoothness score, the higher the probability that the candidate word is the word the user wants to enter, and it can be recommended preferentially. Since the number of candidate words is large, the text smoothness evaluation method of the embodiments of the present disclosure can fully reuse the smoothness evaluation values of some texts cached in the text smoothness cache library, thereby greatly reducing the calculation amount of calculating the smoothness of the text content corresponding to each candidate word, improving the text smoothness calculation efficiency, and further improving the Chinese input efficiency.

[0089] As Figure 2 shown, based on the results obtained by calculating the smoothness of the text content corresponding to multiple candidate words, the multiple candidate words are sorted according to the smoothness score. The candidate words can be displayed in the candidate word display box 212 according to the recommended order. The candidate words preferentially recommended after being evaluated by the text smoothness evaluation method of the embodiments of the present disclosure are: "friend", "pengyou", "pengyou", and "pengyou".

[0090] General description of the embodiments of the present disclosure

[0091] According to an embodiment of the present disclosure, a text smoothness evaluation method is provided. The text smoothness evaluation method provided by the embodiments of the present disclosure can be applied to scenarios such as Figure 2 shown in Chinese input. As Figure 3 shown, it is a schematic flowchart of a text smoothness evaluation method provided by the present disclosure. The text smoothness evaluation method may include:

[0092] Step 310, obtain the text data to be evaluated for smoothness.

[0093] The text data consists of the candidate text corresponding to the string entered by the input object and the associated above content. The text data can be text in any language. The string entered by the input object can be a string entered based on the input method of the corresponding language. For example, if the input object wants to enter the Chinese word "friend", then the corresponding pinyin string "pengyou" can be entered through the input method. The string entered by the input object may correspond to multiple texts, and one of them may be the target text the object wants to enter.

[0094] The candidate text can be text available for the input object to select based on the string input by the input object. For example, if the input object inputs the pinyin string "pengyou", multiple candidate texts that can be obtained based on this pinyin string may include "朋友", "鹏友", "彭友", "捧油", etc. Therefore, in one implementation, before obtaining the text data to be evaluated for fluency, it may further include: obtaining candidate texts based on the string input by the input object.

[0095] In one implementation, obtaining candidate texts based on the string input by the input object includes:

[0096] Performing word segmentation on the string input by the input object to obtain at least one string unit;

[0097] Obtaining at least one candidate text from a preset word library based on the at least one string unit.

[0098] The string input by the input object may contain strings corresponding to multiple characters or words. Therefore, the string input by the input object can be segmented to obtain at least one string unit, and for each string unit, the corresponding character or word can be matched in the preset word library. The candidate text is determined based on the characters or words matched by the at least one string unit in the preset word library. For example, for the string "pengyou" input by the input object, the string units obtained by word segmentation may include "peng" and "you", and then the corresponding words for "peng" and "you" can be matched in the preset word library respectively; the string unit obtained by word segmentation can also be "pengyou", that is, the original string is used as one string unit, and then the word for the string "pengyou" can be matched in the preset word library.

[0099] The preset word library can be pre-created and includes a large amount of word data in the language corresponding to the text data input by the object. In the preset word library, each string unit can be stored corresponding to the character or word found through this string unit. Therefore, the corresponding character or word can be matched in the preset word library based on the string unit. When multiple string units are obtained by word segmentation, the characters or words corresponding to the multiple string units can be combined to obtain the candidate text. For example, the string units include "peng" and "you". The characters or words corresponding to "peng" in the preset word library include "捧", "棚", "朋", etc., and the characters or words corresponding to "you" in the preset word library include "有", "友", "油", etc. Therefore, the candidate texts obtained by combining the words corresponding to "peng" and "you" can include "捧有", "棚有", "朋友", etc.

[0100] In one embodiment, obtaining at least one candidate text based on at least one string unit in a preset word library includes:

[0101] Obtaining multiple reference texts based on at least one string unit in a preset word library;

[0102] Filtering the multiple reference texts based on preset filtering conditions to obtain at least one candidate text.

[0103] The reference texts can be multiple texts directly matched in the preset word library based on at least one string unit. Therefore, the process of obtaining multiple reference texts based on at least one string unit in the preset word library is the same as the process of obtaining at least one candidate text based on at least one string unit in the preset word library in the foregoing embodiment, which will not be elaborated here.

[0104] After obtaining the multiple reference texts, the multiple reference texts can be filtered based on preset filtering conditions to obtain at least one candidate text. Therefore, the candidate text can be a text with a relatively high selection probability among the multiple reference texts.

[0105] In one embodiment, the preset filtering condition can be: the usage frequency of the reference text within a predetermined time period is greater than a predetermined threshold.

[0106] The usage frequency of the reference text can indicate the number of times the reference text has been used within a predetermined time period, and can be stored through text usage records. The text usage records can record the number of times a large amount of text content has been used by different input objects within a predetermined time period before the current time point. Within the predetermined time period, when the input object obtains the target text to be input through the input string, the usage frequency of the target text in the text usage record can be incremented by one.

[0107] The higher the usage frequency corresponding to the reference text in the text usage record, the more frequently the reference text can be represented as being used, and the higher the possibility of being selected. On the contrary, if the usage frequency corresponding to the reference text in the text usage record is very low, it can indicate that the reference text is hardly used, so the possibility of being selected is very low.

[0108] Therefore, the reference text with a usage frequency greater than the predetermined threshold can be determined as the candidate text. For example, the multiple reference texts obtained based on "pengyou" include: "Peng You", "friend", "peng you", "peng you", and "peng you", and the corresponding usage frequencies are: 0, 14, 6, 9, 2, and the predetermined threshold is 5. Therefore, the candidate texts are "friend", "peng you", and "peng you".

[0109] In another implementation, the preset filtering condition may be: the ratio of the frequency of use of the reference text in the input device to the total frequency of use of the reference text within a predetermined time period is greater than a predetermined threshold, or the total frequency of use of the reference text is greater than a predetermined threshold.

[0110] The total usage frequency of the reference text can be the number of times the reference text was used by different input objects within a predetermined time period, as described in the aforementioned embodiments, and will not be repeated here. The input device can be a device through which the input object inputs a string. Therefore, the usage frequency of the reference text on the input device can indicate the number of times the input object uses the reference text through the input device within a predetermined time period. The usage frequency of the reference text on the input device can be obtained based on the device's work log within the predetermined time period.

[0111] The device operation log can record the actions taken by the input device within a predetermined time period prior to the current time, which may include the text content entered by the input object through the electronic device. When the device operation log records that the input object has frequently used a certain reference text, it indicates that the reference text is likely to be used again.

[0112] The ratio of the frequency of reference text usage on the input device to the total frequency of reference text usage within a predetermined time period indicates the specificity of the reference text for the input device. Some reference texts may have a relatively low total usage frequency, meaning their probability of selection is low based solely on total usage frequency. However, they may be used frequently on the input device. In other words, these reference texts are relatively niche for most input objects, but are frequently used on the input device, thus increasing their probability of being selected again. For example, a reference text might have a total usage frequency of 5. The input device's log also shows that this reference text is used 5 times on the input device. This means that all 5 uses of the reference text were performed by the input device. Therefore, the ratio of the frequency of reference text usage on the input device to the total frequency of reference text usage within the predetermined time period is 5 / 5 = 1. Therefore, when the ratio of the frequency of reference text usage on the input device to the total frequency of reference text usage within the predetermined time period exceeds a predetermined threshold, the reference text can be identified as a candidate text.

[0113] Furthermore, when the reference text is used infrequently on the input device but has a high total usage frequency, it indicates that the reference text has high universality. Therefore, even if the reference text is used infrequently on the input device within a predetermined time period, it is still highly likely to be selected later. Thus, when the total usage frequency of the reference text exceeds a predetermined threshold, the reference text can also be identified as a candidate text.

[0114] Using the ratio of the frequency of reference text usage on the input device to the total frequency of reference text usage within a predetermined time period, along with the total frequency of reference text usage, as preset filtering conditions, candidate texts can be determined from two perspectives: the specificity of the reference text for the input device and its universality across multiple input devices, thus improving the accuracy of candidate text determination.

[0115] When a reference text meets the preset filtering criteria, it can be identified as a candidate text. By filtering multiple reference texts obtained from a preset thesaurus based on these criteria, candidate texts can be eliminated, reducing the number of candidate texts and consequently the amount of text data requiring fluency evaluation. This significantly improves the efficiency of candidate text recommendation and, consequently, the efficiency of text input for the target audience.

[0116] The preceding context can be content that the input subject has already entered into the input device and is related to the candidate text. The preceding context and the candidate text can form a coherent text segment without punctuation. For example, if the input subject has already entered "Hello everyone, welcome new" into the input device, and then enters the string "pengyou", then the preceding context for that string is "Welcome new". When the string entered by the input subject is preceded by a tag or has no characters, the preceding context for that string is empty, and the text data can be the candidate text itself.

[0117] In one implementation, after obtaining the candidate text and the associated preamble, the associated preamble and the candidate text can be concatenated into a continuous text content as the text data to be evaluated for fluency. For example, if the associated preamble is "Welcome new," and the candidate text is "Friends," then the text data formed by the associated preamble and the candidate text is "Welcome new friends."

[0118] Step 320: Perform word segmentation on the text data and generate a first text fluency assessment plan based on the word segmentation results.

[0119] To evaluate the fluency of text data, the text data can be segmented to obtain multiple segments, and the fluency of the entire text data can be evaluated based on the fluency between the segments.

[0120] In one implementation, tokenizing text data can tokenize the text data according to the length of text units. When the length of the text unit is 1, each text unit after tokenization contains only the minimum unit elements of the language corresponding to 1 text data. For example, when the text data is in Chinese, the minimum unit element can be a character. Then, tokenizing the text data according to the text unit length of 1 can be performed by dividing it into single characters. For example, if the Chinese text data is "Welcome new friends", the tokenization result obtained by tokenizing the text data can be "欢", "迎", "新", "朋", and "友". Similarly, when tokenizing the above Chinese text data according to the text unit length of 2, each token contains at most two characters, and the tokenization result can be "欢迎", "新朋", and "友". Another example is that when the text data is in English, the minimum unit element can be a word. Then, tokenizing the text data according to the text unit length of 1 can be performed by dividing it into words. For example, if the English text data is "Welcome to my home", then the tokenization result obtained by tokenizing the text data can be "Welcome", "to", "my", and "home". Similarly, when tokenizing the above English text data according to the text unit length of 2, each token contains at most two words.

[0121] In another implementation, tokenizing text data can tokenize the text data according to text phrases. A text phrase can be a phrase composed of the minimum unit elements of the language corresponding to the text data and included in a predetermined vocabulary. The predetermined vocabulary can be corresponding to the language of the text data and is a database for storing fixed collocation words. These fixed collocation words have clear definitions in the corresponding language and can be directly used to express specific meanings. For example, when the text data is in Chinese, tokenizing the text data can be performed by dividing it into Chinese words, and the number of characters in the words can be different. For example, if the Chinese text data is "Welcome new friends", the tokenization result obtained by tokenizing the text data can be "欢迎", "新", and "朋友". Another example is that when the text data is in English, tokenizing the text data can be performed by dividing it into English phrases composed of multiple consecutive words, and the number of words in the phrases can also be different. For example, if the English text data is "Welcome to my new house", then the tokenization result obtained by tokenizing the text data with English phrases can be "Welcome to" and "my new house".

[0122] In one embodiment, tokenizing text data can be performed according to a preset tokenization rule. The preset tokenization rule can be a pre-set tokenization method. For example, tokenizing text data according to a predetermined text unit length. The predetermined text unit length can indicate the length of each token after tokenization. For example, if the predetermined text unit length is 1, then the preset tokenization rule can be to tokenize text data according to a text unit length of 1. Based on this preset tokenization rule, the tokenization result can include multiple tokens of length 1 corresponding to the text data. Using the preset tokenization rule to tokenize text data can improve the efficiency of tokenization.

[0123] In another embodiment, tokenizing text data can be performed based on a preset neural network model. The preset neural network model can be a pre-trained neural network model capable of tokenizing text data based on the meaning of the text. By inputting the text data into the preset neural network model, the preset neural network model can divide the text data into multiple tokens based on the meaning of each word contained in the text data, making the tokenization result more in line with the semantic expression in actual applications. The preset neural network model can be a transformer architecture or a recurrent neural network model, etc. Since the preset neural network model can tokenize text data based on the meaning of the text, this can improve the accuracy of tokenization.

[0124] After tokenizing the text data, a first text fluency evaluation plan can be generated based on the tokenization result. The first text fluency evaluation plan can indicate the order and evaluation method for evaluating the fluency of multiple tokens. Evaluating the fluency of the text is also evaluating the coherence between multiple tokens. Therefore, the token evaluation order in the first text fluency evaluation plan can be the order of multiple tokens in the text data. For example, if the text data is "Welcome new friends", and the tokenization result is "Huan", "Ying", "Xin", "Peng", and "You", then the order of evaluating the fluency of multiple tokens indicated in the first text fluency evaluation plan can also be "Huan", "Ying", "Xin", "Peng", and "You".

[0125] In one embodiment, the evaluation method for evaluating the fluency between multiple tokens can adopt the idea of Bayesian probability. That is to say, the fluency of the text formed by the previous content and the current token can be represented by calculating the probability of the current token appearing under the condition that the previous content appears. For example, the order of evaluating the fluency of multiple tokens is "Huan", "Ying", "Xin", "Peng", and "You". First, since there is no corresponding previous content for the word "Huan", the probability of "Huan" appearing when the previous content is empty can be calculated, that is Next, the probability of "迎" appearing when the previous text is "欢" can be calculated, that is, P('迎'|'欢'); next, the probability of "新" appearing when the previous text is "欢迎" can be calculated, that is, P('新'|'欢迎'); next, the probability of "朋" appearing when the previous text is "欢迎新" can be calculated, that is, P('朋'|'欢迎新'); finally, the probability of "友" appearing when the previous text is "欢迎新朋" can be calculated, that is, P('友'|'欢迎新朋'). The above probability values can represent the coherence degree between the previous text corresponding to the word segmentation and the word segmentation. The higher the probability value, the higher the coherence degree. Therefore, after calculating the coherence degree between each word segmentation, the text smoothness can be obtained based on the coherence degree between multiple word segmentations.

[0126] In one implementation, the smoothness of the text data can be equal to the sum of the coherence degrees corresponding to each word segmentation. That is to say, in the above example, the smoothness corresponding to the text data "欢迎新朋友" is equal to Therefore, the first text smoothness evaluation plan generated based on the word segmentation result of this example can be: SUM( P('迎'|'欢'), P('新'|'欢迎'), P('朋'|'欢迎新'), P('友'|'欢迎新朋')). It can be known from the first text smoothness evaluation plan that: First, when evaluating the smoothness of the text data, the evaluation is carried out in the word segmentation order of "欢", "迎", "新", "朋" and "友"; second, for each word segmentation, the coherence degree between the previous text and the text composed of this word segmentation is evaluated by calculating the probability of this word segmentation appearing under the condition that the previous text appears; finally, the SUM function can represent the summation of the coherence degrees of the texts corresponding to multiple word segmentations. Therefore, when directly executing the first text smoothness evaluation plan, the coherence degree of each word segmentation and the corresponding previous text will be determined in turn according to the word segmentation evaluation order, and the coherence degrees of the texts corresponding to multiple word segmentations will be added up to obtain the smoothness of the text data.

[0127] Step 330, obtain the text smoothness cache library.

[0128] The text smoothness cache library contains multiple cached texts and the smoothness evaluation values corresponding to each cached text. The cached text can represent the text for which the smoothness has been calculated, and the smoothness evaluation value can represent the result obtained after calculating the smoothness of the cached text.

[0129] The text smoothness cache library can be created in advance. That is to say, some alternative text contents are selected in advance, and multiple cached texts are determined based on the alternative text contents, and then the smoothness of each cached text is evaluated to obtain the corresponding smoothness evaluation value. The cached text and the corresponding smoothness evaluation value are stored in the text smoothness cache library.

[0130] In one implementation, the alternative text content is determined through the following process:

[0131] Retrieve the frequency of text usage and the number of objects using each candidate alternative text content within a predetermined time period before the current time point from the text content database.

[0132] Multiple alternative text contents are determined from the text content database based on the frequency of text usage and the number of users of the text.

[0133] The text content database can store a large amount of text content used by multiple input objects, where each piece of text can serve as a candidate for alternative text content. Text usage frequency represents the number of times a corresponding candidate for alternative text content has been used within a predetermined time period prior to the current time point. A higher text usage frequency indicates that the candidate for alternative text content is more frequently used, and therefore more likely to be used again. The number of text-using objects represents the data of input objects that used the corresponding candidate for alternative text content within a predetermined time period prior to the current time point. A larger number of text-using objects indicates that the candidate for alternative text content is more universally applicable, and therefore more likely to be used again by other input objects.

[0134] Text usage frequency and the number of users can be obtained from a text content database. Within a predetermined time period, when a candidate text is used, its usage frequency in the text content database is incremented by 1. When the input object is a new input object that has not previously used the candidate text, the number of users is incremented by 1. The text usage frequency and number of users for each candidate text stored in the text content database can be updated periodically according to a predetermined time frame. This is because some candidate texts may be popular texts within a certain time period, with high usage frequency and a high number of users. However, after a period of time, the popularity of these candidate texts decreases, and both their usage frequency and number of users decrease. Therefore, the text usage frequency and the number of users can be re-counted within this time period to obtain accurate statistical results for the predetermined time period.

[0135] For some candidate alternative text content, the text may be used frequently, but the number of users is small. This indicates that these candidate alternative text contents are frequently used within a specific group of input objects, but are not applicable to other input objects, thus having low universality and a low probability of being reused by most input objects. Similarly, for some candidate alternative text content, the text may be used infrequently, but the number of users is large. This indicates that while these candidate alternative text contents possess universality for use by most input objects, their usage frequency is low, and they may only be usable in specific scenarios, thus having a low probability of being reused. Therefore, a comprehensive evaluation of the text usage frequency and the number of users of candidate alternative text content can be used to determine whether a particular candidate alternative text content can be used as alternative text content.

[0136] In one implementation, multiple alternative text contents are determined from a text content database based on text usage frequency and the number of text users, including:

[0137] The frequency of text usage is input into the first function to obtain the first text score, and the number of objects using the text is input into the second function to obtain the second text score;

[0138] Multiple alternative text contents are determined from the text content database based on the first text score and the second text score.

[0139] The first function can be used to calculate the probability of a candidate alternative text being selected based on the frequency of text usage, which is the first text score; similarly, the second function can be used to calculate the probability of a candidate alternative text being selected based on the number of text users, which is the second text score.

[0140] After calculating the first text score and the second text score, the total text score of the corresponding candidate alternative text content can be calculated based on the first text score and the second text score, and multiple alternative text contents can be determined from the text content database based on the total text score.

[0141] In one implementation, the total text score can be calculated by summing the first text score and the second text score, as shown in Formula 1:

[0142] s = f1(pv) + f2(uv) (Formula 1).

[0143] In Formula 1, s represents the total text score, f1 represents the first function, f2 represents the second function, pv represents the text usage frequency of the candidate alternative text content, and uv represents the number of users of the candidate alternative text content. After calculating the first text score using the first function and the second text score using the second function, the total text score is obtained by summing the first and second text scores. Calculating the total text score by direct summation ensures that the text usage frequency and the number of users contribute equally to the evaluation of the probability of selecting candidate alternative text content, thus improving the fairness of determining alternative text content based on the first and second text scores.

[0144] In another implementation, the total text score can be calculated by weighting the first text score and the second text score, as shown in Formula 2:

[0145] s = w1·f1(pv) + w2·f2(uv) (Formula 2).

[0146] In Formula 2, w1 can represent the first weight corresponding to the first text score, and w2 can represent the second weight corresponding to the second text score. The first and second weights can be used to adjust the impact of text usage frequency and the number of text users on the probability assessment of candidate alternative text content being selected. Therefore, by calculating the total text score through weighted summation, the first and second weights can be adjusted according to the needs of the actual application scenario, improving the flexibility in determining alternative text content.

[0147] In one implementation, multiple backup text contents are determined from a text content database based on the total text score. Candidate backup text contents with a total text score greater than a predetermined threshold are identified as backup text contents. For example, if the predetermined threshold is 5, candidate backup text contents A1 have a total text score of 7, candidate backup text contents A2 have a total text score of 4, candidate backup text contents A3 have a total text score of 6, candidate backup text contents A4 have a total text score of 8, and candidate backup text contents A5 have a total text score of 2. Therefore, the backup text contents determined include: candidate backup text contents A1, candidate backup text contents A3, and candidate backup text contents A4.

[0148] Determining alternative text content based on the frequency of text usage and the number of users of the text can identify alternative text content that is more likely to be reused from the perspective of the usage of each candidate alternative text content, thus improving the accuracy of alternative text content determination.

[0149] After determining the alternate text content, multiple cached texts can be determined based on the alternate text content, and a fluency evaluation is performed for each cached text to obtain the corresponding fluency evaluation value.

[0150] In one implementation, determining multiple cached texts based on the alternate text content and performing a fluency evaluation for each cached text to obtain the corresponding fluency evaluation value includes:

[0151] Perform word segmentation on the alternate text content, and obtain multiple cached texts based on the word segmentation result;

[0152] Perform a fluency evaluation for each cached text to obtain the corresponding fluency evaluation value.

[0153] The process of performing word segmentation on the alternate text content can be similar to the process of performing word segmentation on text data in the foregoing implementation, and will not be elaborated here.

[0154] After obtaining multiple word segments corresponding to the alternate text content, it can be extended step by step from the first word segment backward to obtain multiple cached texts. That is to say, the cached texts include the first word segment in the alternate text content and a predetermined number of word segments after the first word segment. For example, for the alternate text content "Welcome", the result of word segmentation is: "Huan", "Ying", "Guang", and "Lin". The first word segment can be used as a cached text alone. Extending the first word segment one step backward can obtain the cached text "Welcome". Extending the first word segment two steps backward can obtain the cached text "Welcome Guan". Extending the first word segment three steps backward can obtain the cached text "Welcome Guanglin". In this way, multiple cached texts corresponding to the alternate text content are obtained, and then a fluency evaluation can be performed for each cached text to obtain the corresponding fluency evaluation value.

[0155] The fluency evaluation of the cached text can adopt the text fluency evaluation method of the embodiments of the present disclosure. However, when creating a text fluency cache library based on the alternate text content, the text fluency cache library is empty, so the existing cached texts and fluency evaluation values in the text fluency cache library cannot be reused. Therefore, the fluency evaluation of the cached text can be performed by executing the first text fluency evaluation plan corresponding to the cached text. The first text fluency evaluation plan has been described in detail in step 320 and will not be elaborated here.

[0156] In one implementation, executing the first text fluency evaluation plan to perform a fluency evaluation on the corresponding cached text can be through a preset neural network model.

[0157] The preset neural network model can be a large language model, which can be used to execute the first text fluency evaluation plan to obtain the fluency evaluation value corresponding to the cached text.

[0158] In one of the aforementioned embodiments, the first text fluency evaluation plan corresponding to the cached text is generated based on the word segmentation results after word segmentation processing of the cached text. The first text fluency evaluation plan can indicate the order and method of fluency evaluation for multiple word segments corresponding to the cached text. Therefore, the preset neural network model can evaluate each evaluation step according to a predetermined evaluation method based on the first text fluency evaluation plan.

[0159] In one of the aforementioned embodiments, the first text fluency evaluation plan indicates that fluency evaluation can be performed on the coherence between each word and its corresponding preceding text. Therefore, the preset neural network model can calculate the coherence of each word and finally obtain the fluency evaluation value of the entire cached text based on the coherence of multiple words.

[0160] The process of the preset neural network model performing fluency assessment based on the first text fluency assessment plan can be represented as follows: Figure 4 Tokenization of the cached text yields n tokens, namely token S1, token S2, ..., token Sn. Based on the first text fluency evaluation plan, the coherence of each token and its corresponding preceding text is calculated sequentially. For example... Figure 4 As shown, firstly, word segmentation S1 is input into the preset neural network model. The preset neural network model can calculate the probability P(segmentation S1|empty) corresponding to word segmentation S1 when the preceding text is empty. The preset neural network model can retain the model state parameter k1 and pass the model state parameter k1 to the preset neural network model used to calculate the next word segmentation. The model state parameter can indicate the model state of the preset neural network model after calculating the coherence between the word segmentation and its corresponding preceding text. Passing the model state parameter to the preset neural network model used to calculate the next word segmentation ensures that the preset neural network model used to calculate the next word segmentation can obtain the coherence information of the preceding text. Next, word segmentation S1 and word segmentation S2 are input into the preset neural network model. The preset neural network model can calculate the probability P(segmentation S2|segmentation S1) corresponding to word segmentation S2 when the preceding text is word segmentation S1. At the same time, the model state parameter k2 is retained and passed to the preset neural network model used to calculate the next word segmentation. Similarly, finally, the segmented words S1 to Sn are input into a preset neural network model. This model calculates the probability P(segment Sn|segment S1…segment Sn-1) corresponding to segment Sn when the preceding text is segmented from S1 to Sn-1. After calculating the coherence between each segmented word and its corresponding preceding text in the cached text, the overall fluency evaluation value of the cached text can be obtained. In one implementation, the coherence between each segmented word and its corresponding preceding text can be summed to obtain the overall fluency evaluation value of the cached text.

[0161] In another implementation, multiple cached texts are determined based on the backup text content, and a fluency evaluation is performed for each cached text to obtain the corresponding fluency evaluation value, including:

[0162] Perform a fluency evaluation on the backup text content;

[0163] Based on the intermediate process of the fluency evaluation, multiple cached texts and the corresponding fluency evaluation value for each cached text are obtained.

[0164] The process of performing a fluency evaluation on the backup text content is the same as the method of performing a fluency evaluation on the cached text in the foregoing implementation. First, the backup text content is segmented, and a first text fluency evaluation plan corresponding to the backup text content is generated based on the segmentation result; then, the fluency evaluation of the backup text content is performed by executing the first text fluency evaluation plan corresponding to the backup text content.

[0165] The process of segmenting the backup text content can be similar to the process of segmenting the text data in the foregoing implementation, which will not be elaborated here. After generating the first text fluency evaluation plan corresponding to the backup text content, the fluency evaluation result corresponding to the backup text content can be obtained by executing the first text fluency evaluation plan through a preset neural network model. The process of using the preset neural network model to execute the first text fluency evaluation plan corresponding to the backup text content is the same as the process of using the preset neural network model to execute the first text fluency evaluation plan corresponding to the cached text in the foregoing implementation, which will not be elaborated here.

[0166] Therefore, after using the preset neural network model to execute the first text fluency evaluation plan corresponding to the backup text content, the fluency evaluation result corresponding to the backup text content can be obtained. During the intermediate process of executing the first text fluency evaluation plan, the coherence degree between each segmented word in the backup text content and the corresponding previous text content can also be obtained. For example, if the backup text content is "Welcome", the corresponding segmentation result and evaluation order are "Huan", "Ying", "Guang", and "Lin". Therefore, when using the preset neural network model to execute the first text fluency evaluation plan, the probability corresponding to "Huan" when the previous text is empty can be calculated first; then, the probability of "Ying" when the previous text is "Huan" can be calculated; then, the probability of "Guang" when the previous text is "Welcome" can be calculated; finally, the probability of "Lin" when the previous text is "Welcome Guang" can be calculated.

[0167] After the completion of the first text fluency evaluation plan, the alternative text content can be used as a cached content, and the fluency evaluation value corresponding to the alternative text content can be obtained based on the coherence degree between each word segment and the corresponding previous text content. For example, the fluency evaluation value corresponding to "Welcome" is the sum of the probability values of connecting each word segment with the previous text content. Similarly, the text obtained by connecting each word segment with the corresponding previous text content can be used as a cached text. For each cached text, the fluency evaluation value corresponding to the cached text can be obtained based on the probability values corresponding to the corresponding word segment and its previous word segment. For example, for the alternative text content "Welcome", "欢" can be used as a cached text alone, and its corresponding fluency evaluation value is the probability of "欢" when the previous text is empty; "欢迎" can be used as a cached text alone, and its corresponding fluency evaluation value is equal to the sum of the probability of "欢" when the previous text is empty and the probability of "迎" when the previous text is "欢"; "欢迎光" can be used as a cached text alone, and its corresponding fluency evaluation value is equal to the sum of the probability of "欢" when the previous text is empty, the probability of "迎" when the previous text is "欢", and the probability of "光" when the previous text is "欢迎". In this way, by executing the first text fluency evaluation plan corresponding to the alternative text content "Welcome", the fluency evaluation values corresponding to the cached texts "欢", "欢迎", "欢迎光", and "Welcome" can be obtained.

[0168] To evaluate the fluency of the alternative text content and obtain multiple cached texts and the fluency evaluation value corresponding to each cached text based on the intermediate process of the fluency evaluation, it is not necessary to separately evaluate the fluency of each cached text included in the alternative text content. Instead, the fluency of each cached text included in the alternative text content can also be evaluated during the process of evaluating the fluency of the alternative text content. In this way, the efficiency of constructing the text fluency cache library is improved.

[0169] After obtaining the fluency evaluation value corresponding to the cached text, the cached text and the fluency evaluation value can be stored in the text fluency cache library in a corresponding manner. In one implementation, the cached text, the final model state parameters of the preset neural network model for calculating the fluency evaluation value of the cached text, and the fluency evaluation value can be stored in the text fluency cache library in a corresponding manner. In this way, when using the preset neural network model and the cached texts in the text fluency cache library to evaluate the text fluency of the text data to be evaluated, the model state parameters corresponding to the cached text can be obtained, and thus the fluency information corresponding to the cached text can be obtained.

[0170] In one implementation, the text fluency cache library can store the cached text and the corresponding fluency evaluation value in the form of a comparison table. For example Figure 5As shown, the fluency evaluation value corresponding to the cached text "欢" is 0.34; the fluency evaluation value corresponding to "欢迎" is 0.67; the fluency evaluation value corresponding to "欢迎光" is 0.85; the fluency evaluation value corresponding to "欢迎光临" is 0.93. Storing the cached text and the corresponding fluency evaluation values in the form of a control relation table can improve the efficiency of finding the cached text in the text fluency cache library. When it is necessary to store the model state parameters corresponding to the cached text in the text fluency cache library, the model state parameters can also be stored at the storage location corresponding to the cached text.

[0171] In another implementation, the text fluency cache library can store the cached text and the corresponding fluency evaluation values in the structure of a trie tree. The trie tree can store multiple cached texts in the form of tree node paths according to the character arrangement order. For example, the cached texts include: "欢", "欢迎", "欢迎光", and "欢迎光临". Then, the storage method of these four cached texts in the text fluency cache library can be as Figure 6 shown: "欢" can be used as the root node, "迎" can be used as the child node of "欢", "光" can be used as the child node of "迎", and "临" can be used as the child node of "光".

[0172] When multiple cached texts have the same prefix, multiple cached texts can be used as the child nodes of the node corresponding to the prefix text at the same time. For example, in addition to the above four cached texts, the cached texts also include: "欢迎来", "欢迎来到", "欢喜", and "欢迎新". Among them, "欢喜" and "欢迎" have the same prefix "欢". Therefore, "喜" and "迎" can be used as the child nodes of "欢"; "欢迎来", "欢迎新", "欢迎来到", and "欢迎光" have the same prefix "欢迎". Therefore, "来", "新", and "光" can be used as the child nodes of "迎" at the same time, and the prefix text of "到" is "欢迎来", and there is no cached text using this prefix text in other cached texts. Therefore, "到" can be used as the child node of "来" alone. The storage method of the above multiple cached texts in the text fluency cache library can be as Figure 7 shown.

[0173] Each node in the trie tree stores the node data in the structure of a key-value pair. Among them, the key can be the character in the cached text.

[0174] In one implementation, the value can include the fluency evaluation value of the cached text corresponding to the character. When it is necessary to store the model state parameters corresponding to the cached text in the text fluency cache library, the value can also include the model state parameters corresponding to the cached text corresponding to the character. For example, in Figure 6In the trie shown, the value corresponding to the node "欢" can include the fluency evaluation value corresponding to the cached text "欢" and the model state parameters; the value corresponding to the node "迎" can include the fluency evaluation value corresponding to the cached text "欢迎" and the model state parameters; the value corresponding to the node "光" can include the fluency evaluation value corresponding to the cached text "欢迎光" and the model state parameters; the value corresponding to the node "临" can include the fluency evaluation value corresponding to the cached text "欢迎光临" and the model state parameters.

[0175] In another implementation, the value can be the storage location of the fluency evaluation value of the cached text corresponding to the character. When it is necessary to store the model state parameters corresponding to the cached text into the text fluency cache library as well, the value can be the storage location of the fluency evaluation value of the cached text corresponding to the character and the model state parameters. In this way, the fluency evaluation value corresponding to the corresponding cached text can be obtained by accessing the storage location corresponding to the node.

[0176] Storing the cached text and the corresponding fluency evaluation values in the text fluency cache library in the form of a trie can greatly reduce the amount of data stored in the text fluency cache library, increase the number of cached texts that the text fluency cache library can store, make it easier for the text data to be processed to match the cached texts in the text fluency cache library, thereby improving the reuse rate of the cached texts in the text fluency cache library and the text fluency evaluation efficiency.

[0177] After creating the text fluency cache library, when generating the first text fluency evaluation plan corresponding to the text data, the text fluency cache library can be obtained.

[0178] In one implementation, the text fluency cache library can be integrated in an electronic device that executes the text fluency evaluation method of the present disclosure.

[0179] In another implementation, the text fluency cache library can be independent of the electronic device that executes the text fluency evaluation method of the present disclosure. When it is necessary to evaluate the fluency of text data, the text fluency cache library can be accessed through a wired connection or a communication connection.

[0180] In one implementation, obtaining the text fluency cache library includes:

[0181] Obtaining the text identifier corresponding to the text data;

[0182] Determining the text fluency cache library corresponding to the text data based on the text identifier in multiple candidate text fluency cache libraries.

[0183] The amount of cached text data can be quite large. Therefore, multiple candidate text fluency cache libraries can be created to store different cached texts in blocks. Each candidate text fluency cache library can cache specific cached text based on its text identifier. The text identifier can be a label used to represent coarse information about the cached text. Even long cached texts can be distinguished using text identifiers. Since the purpose of the text identifier is to roughly classify the cached text, different cached texts may have the same text identifier.

[0184] In one implementation, the text identifier can be determined based on the Unicode value of the first character in the cached text. Unicode is a character set that provides unique numerical codes for the script, symbols, and punctuation of various languages. Therefore, the Unicode value corresponding to each character is unique. After determining the Unicode value corresponding to the first character, the remainder of the Unicode value divided by a predetermined value can be used to obtain the text identifier corresponding to the cached text.

[0185] Different candidate text fluency caching libraries can store corresponding cached text based on specific text identifiers or specific ranges of text identifiers. For example... Figure 8 As shown, candidate text fluency cache D1 stores cached text with a text identifier equal to 0; candidate text fluency cache D2 stores cached text with a text identifier equal to 1; and candidate text fluency cache Dn stores cached text with a text identifier equal to n-1. When evaluating the fluency of text data, a text fluency cache can be obtained from multiple candidate text fluency caches based on the text identifier corresponding to the text data. For example, if the text identifier corresponding to the text data is 1, then its corresponding text fluency cache is candidate text fluency cache D2.

[0186] Assigning multiple cached texts to different candidate text fluency cache libraries based on text identifiers can achieve block storage of a large number of cached texts, which is beneficial to improving the matching efficiency of cached texts for matching text data.

[0187] Step 340: Based on the comparison results between the cached text and the text data, revise the first text fluency evaluation plan to obtain the second text fluency evaluation plan.

[0188] The comparison results include the matched text and its corresponding fluency score. The matched text is a portion of the text data that can be matched against a cached text in the text fluency cache. The fluency score is the fluency evaluation value of the matched text in the text fluency cache.

[0189] When the comparison result is empty, that is, when a partial text in the text data does not exist in multiple cached texts, the first text fluency evaluation plan may not be corrected, and then the first text fluency evaluation plan may be directly executed to evaluate the text fluency corresponding to the text data.

[0190] When the comparison result is not empty, that is, when a partial text in the text data exists in multiple cached texts, the first text fluency evaluation plan may be corrected according to the comparison result to obtain a second text fluency evaluation plan.

[0191] In one implementation, correcting the first text fluency evaluation plan according to the comparison result between the cached text and the text data to obtain a second text fluency evaluation plan includes:

[0192] Compare the text data with the cached texts in the text fluency cache library, and determine the target cached text in the text fluency cache library according to the comparison result;

[0193] Determine at least one redundant step in multiple first evaluation steps included in the first text fluency evaluation plan based on the target cached text;

[0194] Correct the first text fluency evaluation plan according to at least one redundant step to obtain a second text fluency evaluation plan.

[0195] After comparing the text data with the cached texts in the text fluency cache library, when a partial text in the cached text includes the partial text in the text data, the partial text may be determined as the target cached text. For example, the text fluency cache library includes "huan", "welcome", "welcome to", and "welcome to visit", and the text data is "welcome to". Since "welcome" is included in multiple cached texts, "welcome" may be used as the target cached text.

[0196] In one implementation, comparing the text data with the cached texts in the text fluency cache library and determining the target cached text in the text fluency cache library according to the comparison result includes:

[0197] Sort multiple word segments determined by the word segmentation result in the text order in the text data, and generate multiple first consecutive phrases including source word segments based on the sorting order and multiple word segments;

[0198] Compare multiple first consecutive phrases with the cached texts in the text fluency cache library to obtain multiple matching second consecutive phrases;

[0199] Determine the consecutive phrase with the largest number of word segments included in multiple second consecutive phrases as the target cached text.

[0200] The source token is the token sorted first. For example, if the tokenization results of the text data "Welcome" are "Huan", "Ying", "Guang", and "Lin", then the first token "Huan" is the source token. Starting from the source token, multiple first consecutive phrases containing different numbers of tokens are obtained according to the text order of the tokens in the text data. For example, if the source token is "Huan", based on the above text order, the first consecutive phrases that can be obtained may include "Huan" (one token), "Huanying" (two tokens), "Huanyingguang" (three tokens), and "Huanyingguanglin" (four tokens).

[0201] After generating multiple first consecutive phrases, each first consecutive phrase can be compared with the cached texts in the text fluency cache library. When the multiple cached texts include a first consecutive phrase, the first consecutive phrase can be determined as the second consecutive phrase that matches successfully. When there are multiple second consecutive phrases, the consecutive phrase with the largest number of tokens can be determined as the target cached text. For example, the text fluency cache library includes "Huan", "Huanying", "Huanyingguang", and "Huanyingguanglin", and the text data is "Welcome to", and its corresponding first consecutive phrases include "Huan", "Huanying", "Huanyinglai", and "Huanyinglaidao". Among them, "Huan" and "Huanying" are successfully matched in the multiple cached texts and can be determined as the second consecutive phrases. And "Huanying" contains the largest number of tokens, so "Huanying" can be determined as the target cached text.

[0202] Compare multiple first consecutive phrases containing different numbers of tokens with the cached texts in the text fluency cache library respectively, and determine the one with the largest number of tokens among the multiple second consecutive phrases obtained by matching as the target cached text. In this way, more redundant steps can be determined in the first text fluency evaluation plan, improving the efficiency of subsequent execution of the second text fluency evaluation plan.

[0203] In one implementation, comparing multiple first consecutive phrases with the cached texts in the text fluency cache library can be done by comparing each of the multiple first consecutive phrases with the cached texts in the text fluency cache library one by one in the order of the number of tokens contained in the multiple first consecutive phrases from more to less. In this way, when a first consecutive phrase is successfully matched in the multiple cached texts, even if the subsequent first consecutive phrases to be compared have not been compared yet, it can be determined that they are successfully matched in the multiple cached texts, and this first consecutive phrase contains the largest number of tokens, so it can be directly determined as the target cached text.

[0204] For example, the text fluency cache library includes "欢", "欢迎", "欢迎光", and "欢迎光临". The text data is "欢迎光顾", and its corresponding first consecutive phrases include "欢", "欢迎", "欢迎光", and "欢迎光顾". Arranged in descending order of the number of segmented words, they are: "欢迎光顾", "欢迎光", "欢迎", and "欢". According to the above arrangement order, first compare "欢迎光顾" in multiple cached texts, and there is no comparison result; then compare "欢迎光" in multiple cached texts, and the comparison is successful. At this point, since the comparison of "欢迎光" is successful, the subsequent "欢迎" and "欢" can be determined to be successful in multiple cached texts even though they have not been compared. And "欢迎光", as the second consecutive phrase with the most segmented words, can be directly determined as the target cached text.

[0205] It can be seen that by comparing the first consecutive phrases one by one in descending order of the number of segmented words, the first second consecutive phrase obtained from the comparison can be directly used as the target cached text, without comparing the subsequent first consecutive phrases, which greatly improves the comparison efficiency.

[0206] In the foregoing embodiment, the text fluency cache library can store the cached texts and the corresponding fluency evaluation values in the structure of a trie. The trie can store multiple cached texts in the form of tree node paths according to the character arrangement order. Based on this, in another embodiment, comparing multiple first consecutive phrases with the cached texts in the text fluency cache library can be performed by comparing layer by layer in the trie in ascending order of the number of segmented words included in the multiple first consecutive phrases.

[0207] Comparing layer by layer in the trie can be as follows: Compare the first first consecutive phrase, that is, the source segmented word, in the first layer of the trie; after the comparison is successful, compare the second segmented word after the source segmented word in the second first consecutive phrase in the child node corresponding to the comparison result of the source segmented word; after the comparison is successful again, compare the third segmented word after the second segmented word in the third first consecutive phrase in the child node corresponding to the comparison result of the second segmented word, and so on. When the comparison fails, even if the subsequent first consecutive phrases have not been compared, it can be determined that the comparison fails. At this time, the first consecutive phrase corresponding to the node where the previous layer's comparison was successful can be determined as the target cached text. Since the comparison is performed in ascending order of the number of segmented words included, the last first consecutive phrase with a successful comparison can be directly determined as the target cached text.

[0208] For example, the text fluency cache library stored in the trie structure can be as Figure 7As shown, the text data is "Welcome back". Its corresponding first consecutive phrases include "Huan", "Welcome", "Welcome back", and "Welcome back". This is also the result of arranging them in ascending order of the number of segmented words. Compare the first consecutive phrases in the above order one by one: First, compare "Huan" at the first layer of the trie tree; after successful comparison, compare "Ying" in "Welcome" in the child nodes corresponding to the node of "Huan"; after successful comparison, compare "Hui" in "Welcome back" in the child nodes corresponding to the node of "Ying", and the comparison fails. When the comparison of "Welcome back" fails, there is no need to compare "Welcome back" anymore. "Welcome" can be directly determined as the target cached text as the last first consecutive phrase that was successfully compared among multiple first consecutive phrases.

[0209] The text fluency cache library stored in the form of trie tree results can greatly reduce the storage space of cached text. At the same time, based on the trie tree, compare multiple first consecutive phrases one by one according to the hierarchical relationship between nodes. When the comparison fails, it can stop in time and obtain the last first consecutive phrase that was successfully compared as the target cached text. In this way, the comparison efficiency can also be greatly improved.

[0210] After determining the target cached text, at least one redundant step can be determined based on the target cached text among multiple first evaluation steps included in the first text fluency evaluation plan. The first evaluation step can indicate the token evaluation order of the tokenization result corresponding to the text data in the first text fluency evaluation plan, which has been described in detail in the foregoing embodiments and will not be elaborated here.

[0211] The redundant step can be the evaluation step corresponding to the target cached text among multiple first evaluation steps. Since the text fluency cache library stores the text fluency evaluation result of the target cached text, in the first text fluency evaluation plan, the evaluation step corresponding to the target cached text belongs to the redundant step that does not need to be executed anymore.

[0212] In one embodiment, determining at least one redundant step based on the target cached text among multiple first evaluation steps included in the first text fluency evaluation plan includes:

[0213] Obtain the evaluation target corresponding to each first evaluation step in the first text fluency evaluation plan;

[0214] Determine the first evaluation step where the evaluation text is consistent with the target cached text as the matching step;

[0215] Based on the execution order between multiple first evaluation steps in the first text fluency evaluation plan, determine the pre-step of the matching step, and determine at least one redundant step according to the pre-step and the matching step.

[0216] The evaluation objectives include the evaluation text and the evaluation fluency value. The evaluation text can be the text that needs to be evaluated for fluency in this first evaluation step; the evaluation fluency value can indicate the way to evaluate the fluency of the text. For example, in the first text fluency evaluation plan corresponding to the text data "Welcome", the word segmentation evaluation steps are successively "欢", "迎", "光", and "临". For the first evaluation step of "欢", in its corresponding evaluation objective, the evaluation text is "欢", and the evaluation fluency value is For the first evaluation step of "迎", in its corresponding evaluation objective, the evaluation text is "欢迎", and the evaluation fluency value is P('迎'|'欢'); for the first evaluation step of "光", in its corresponding evaluation objective, the evaluation text is "欢迎光", and the evaluation fluency value is P('光'|'欢迎'); for the first evaluation step of "临", in its corresponding evaluation objective, the evaluation text is "欢迎光临", and the evaluation fluency value is P('临'|'欢迎光').

[0217] Determine the first evaluation step in the evaluation text that is the same as the target cache text as the matching step. For example, if the target cache text is "欢迎", then in the above first evaluation step, the first evaluation step for "迎" can be determined as the matching step.

[0218] The execution order between multiple first evaluation steps can be pre-determined and can be consistent with the arrangement order of the word segmentations corresponding to the multiple first evaluation steps in the text data. In this way, the pre-step of the matching step can be the first evaluation step corresponding to the word segmentation before the word segmentation corresponding to the matching step. For example, if the matching step is the first evaluation step corresponding to "迎", its pre-step can be the first evaluation step corresponding to "欢". When there is no pre-step before the matching step, then the matching step can be determined as a redundant step; when there is a pre-step before the matching step, then the matching step and all pre-steps can be determined as redundant steps.

[0219] Determining both the matching step and the pre-step of the matching step as redundant steps can ensure that the evaluation result corresponding to the target cache text can be fully reused, thereby improving the accuracy of subsequent correction of the first text fluency evaluation plan.

[0220] After determining the redundant steps, the first text fluency evaluation plan can be corrected based on the redundant steps to obtain the second text fluency evaluation plan.

[0221] In one implementation manner, correcting the first text fluency evaluation plan according to at least one redundant step to obtain the second text fluency evaluation plan includes:

[0222] Delete the evaluation steps in the first text fluency evaluation plan that are the same as the redundant steps to obtain multiple second evaluation steps;

[0223] The second text fluency assessment plan is generated based on the execution order of multiple second assessment steps and the steps in the first text fluency assessment plan.

[0224] The first text fluency assessment plan includes multiple assessment steps, including redundant steps. Therefore, the redundant steps can be removed from the first text fluency assessment plan, retaining only the multiple second assessment steps. Then, based on the execution order of the multiple second assessment steps in the first text fluency assessment plan, a second text fluency assessment plan is generated.

[0225] For example, the first text fluency assessment plan is SUM( P('Welcome'|'Happy'), P('New'|'Welcome'), P('Friend'|'Welcome New'), P('Friend'|'Welcome New Friend')), the redundant steps are the evaluation steps for "Happy". And the evaluation step P('welcome'|'happy') for "welcome". Therefore, after removing redundant steps, the second text fluency evaluation plan generated based on the remaining second evaluation steps can be SUM(P('new'|'welcome'), P('friend'|'welcome new'), P('friend'|'welcome new friend')).

[0226] By removing redundant steps from the first text fluency assessment plan and generating a second text fluency assessment plan based on the remaining steps in the order of execution of the steps in the first text fluency assessment plan, the accuracy of generating the second text fluency assessment plan based on redundant steps can be improved.

[0227] Modifying the first text fluency evaluation plan based on redundant steps ensures that evaluation steps that have already matched cached text will not be executed when the second text fluency evaluation plan is executed, thus improving the accuracy of the first text fluency evaluation plan modification.

[0228] Step 350: Execute the second text fluency assessment plan, and calculate the second text fluency corresponding to the text data based on the first text fluency and matching fluency obtained from the second text fluency assessment plan.

[0229] The process of executing the second text fluency assessment plan can be the same as the process of executing the first text fluency assessment plan corresponding to the cached text in the aforementioned embodiments, and will not be described again here.

[0230] After executing the second text fluency assessment plan, a first text fluency score corresponding to the second text fluency assessment plan can be obtained. Since the second text fluency assessment plan is based on a modification of the first text fluency assessment plan, the first text fluency score is the fluency assessment result for text content other than the matching text in the text data. Therefore, the second text fluency score corresponding to the text data can be calculated based on the first text fluency score and the matching fluency score corresponding to the matching text. In one implementation, the second text fluency score is equal to the sum of the first text fluency score and the matching fluency score.

[0231] In one of the aforementioned embodiments, the execution of the first text fluency evaluation plan to evaluate the fluency of the corresponding cached text can be achieved through a preset neural network. Based on this, in one embodiment, the text fluency cache library further includes model state parameters corresponding to each cached text, the model state parameters indicating the model state of the preset neural network model after calculating the fluency evaluation value corresponding to the cached text.

[0232] Therefore, implementing the second text fluency assessment plan may include:

[0233] Retrieve the matching model state parameters corresponding to the matched text from the text fluency cache library;

[0234] The preset neural network model is updated based on the matching model state parameters, and the second text fluency evaluation plan is executed using the updated preset neural network model.

[0235] In some embodiments, a pre-defined neural network model can be used to evaluate the fluency of the text to be evaluated. This pre-defined neural network model can be called a text fluency evaluation model. When using the text fluency evaluation model to evaluate the fluency of the text to be evaluated, the text can first be segmented into multiple words, and then the fluency evaluation model can be applied. The specific process of using the fluency evaluation model to evaluate the fluency of the text to be evaluated is described in [the following text is incomplete and requires further context]. Figure 4 As already described, it will not be repeated here. In this embodiment of the disclosure, in addition to caching the cached text and the corresponding fluency evaluation value, the text fluency cache library also caches the model state parameters corresponding to each cached text. These model state parameters may include, but are not limited to, the model parameters of the fluency evaluation model and the intermediate model parameters output by the evaluation model after previously evaluating the fluency of the cached text.

[0236] Since the second text fluency assessment plan involves evaluating the fluency of text content other than the matching text in the text data, updating the preset neural network model using the matching model's state parameters allows the preset neural network model to perform fluency assessments based on the fluency assessment parameters of the matching text when executing the second text fluency assessment plan. Specifically, updating the preset neural network model using the matching model's state parameters sets the input parameters to the aforementioned intermediate parameters and the model's own parameters to the parameters corresponding to the state after the last fluency assessment of the cached text. This way, even if the matching text and the text content other than the matching text in the text data are evaluated for fluency separately, the two text contents can be correlated when executing the second text fluency assessment plan using the preset neural network model, which helps improve the accuracy of fluency assessment of the text data.

[0237] In one implementation, after executing a second text fluency assessment plan and calculating a second text fluency corresponding to the text data based on the first text fluency assessed by executing the second text fluency assessment plan and the matching fluency, the method further includes:

[0238] Based on the text data and the matching text, at least one text to be stored is determined, and the fluency evaluation value corresponding to each text to be stored is obtained based on the execution process of the second text fluency evaluation plan.

[0239] The text fluency cache is updated based on the text to be stored and its corresponding fluency evaluation value.

[0240] The text to be stored can be text content not included in the text fluency cache library. Therefore, the text to be stored included in the text data can be determined based on the matching text. For example, if the text data is "Welcome", and the matching text is "Welcome", then the text to be stored can include both "Welcome" and "Welcome".

[0241] During the execution of the second text fluency assessment plan, the coherence between each word segment and the preceding text can be evaluated individually. Based on the coherence of each word segment and the preceding text, the first text fluency score can be obtained. Therefore, during the execution of the second text fluency assessment plan, the coherence between each word segment to be evaluated and the preceding text can be obtained. Based on the coherence of the word segment and the fluency assessment value of the preceding prefix word, the fluency assessment value to be stored for the text formed by the word segment and the preceding text can be obtained.

[0242] For example Figure 9As shown, the text data is "Welcome", and the matching text is "Welcome". During the process of executing the second text smoothness evaluation plan, the coherence degree between "Welcome" and "Light" can be calculated, and based on this coherence degree and the matching smoothness degree corresponding to "Welcome", the smoothness evaluation value to be stored corresponding to "Welcome Light" can be obtained. Secondly, the coherence degree between "Welcome Light" and "Come" can be calculated, and based on this coherence degree and the smoothness evaluation value to be stored corresponding to "Welcome Light", the smoothness evaluation value to be stored corresponding to "Welcome" can be obtained.

[0243] After determining the text to be stored and the corresponding smoothness evaluation value to be stored, the text to be stored and the corresponding smoothness evaluation value to be stored can be added to the text smoothness cache library.

[0244] The process of adding the text to be stored and the corresponding smoothness evaluation value to be stored to the text smoothness cache library is the same as the process of adding the cached text and the corresponding smoothness evaluation value to the text smoothness cache library in the foregoing embodiment.

[0245] In one embodiment, the text smoothness cache library can store the cached text and the corresponding smoothness evaluation value in the form of a comparison relation table. In this way, the text to be stored and the smoothness evaluation value to be stored can be stored correspondingly in the text smoothness cache library.

[0246] In another embodiment, the text smoothness cache library can store the cached text and the corresponding smoothness evaluation value in the structure of a trie tree. In this way, when there is a prefix text in the text to be stored in the trie tree, the text after the prefix text can be stored based on the branch node corresponding to the prefix text. The branch node can be the node corresponding to the last character in the prefix text. For example, the text to be stored is "Welcome Light", and the prefix text included in the text smoothness cache library is "Welcome", so the branch node is the node corresponding to "Welcome", and "Light" can be stored as the child node of "Welcome". Similarly, when storing the text to be stored "Welcome", "Come" can be stored as the child node of "Light".

[0247] Updating the text smoothness cache library based on the text to be stored and the corresponding smoothness evaluation value to be stored can enable the text smoothness cache library to continuously add new cached text and the corresponding smoothness evaluation value, so that the cached text in the text smoothness cache library is more likely to hit the text data to be evaluated for smoothness, which is beneficial to further improving the text smoothness evaluation efficiency.

[0248] In one embodiment, the text smoothness evaluation method of the embodiments of the present disclosure further includes:

[0249] When the text fluency cache reaches a predetermined condition, obtain the usage frequency and most recent usage time of each cached text in the text fluency cache.

[0250] The text fluency cache library is filtered based on usage frequency and recent usage time.

[0251] Predefined conditions can indicate a predetermined cache filtering period. The text fluency cache library can perform cache filtering according to the predetermined period. Therefore, when the cache filtering time is reached, the usage frequency and most recent usage time of each cached text in the text fluency cache library can be obtained.

[0252] Another pre-defined condition is when the memory occupied by cached text in the text fluency cache reaches a predetermined value. For example, when the memory occupied by cached text reaches 70% of the total content of the text fluency cache, cache filtering can be performed.

[0253] The usage frequency of cached text represents the number of times a cached text is used within a predetermined time period prior to the cache filtering time. The usage frequency can be stored in the corresponding storage location of the cached text in the text fluency cache library. When the text fluency cache library is initially created, the usage frequency for each cached text is 0. When a text to be stored in the aforementioned implementation is added to the text fluency cache library, its corresponding usage frequency can be initialized to 1. When a cached text is reused, its corresponding usage frequency can be incremented by 1. After performing cache filtering on the text fluency cache library, the usage frequency for each cached text can be initialized to 0. A higher usage frequency for a cached text indicates a greater probability that the cached text will be reused.

[0254] The last used time (UUT) of cached text represents the time when the cached text was last used. UUT can also be stored in the corresponding storage location of the cached text in the text fluency cache library. When initially creating the text fluency cache library, the UUT for each cached text can be empty. When adding text to be stored in the aforementioned implementation to the text fluency cache library, its corresponding UUT can be the time when the text data's fluency was evaluated. When cached text is reused, the UUT can be updated to the reuse time. The closer the UUT of a cached text is to the cache filtering time, the greater the probability that the cached text will be reused again in the near future.

[0255] After determining the usage frequency and the most recent usage time, multiple cached texts in the text fluency cache library can be filtered based on the usage frequency and the most recent usage time.

[0256] In one implementation, filtering multiple cached texts in a text fluency cache library based on usage frequency and recent usage time includes:

[0257] Get the effective text length of each cached text;

[0258] The text fluency cache library is filtered based on usage frequency, recent usage time, and effective text length.

[0259] The effective text length of the cached text indicates the text length that needs to be calculated when evaluating the fluency of the cached text. The longer the effective text length, the greater the computational load and the higher the computational cost when evaluating the fluency of the cached text.

[0260] In one implementation, the effective text length of the cached text can be equal to the text length of the cached text.

[0261] In one of the aforementioned embodiments, the text fluency cache library stores cached text in a trie structure. Based on this, in another embodiment, obtaining the effective text length of each cached text includes:

[0262] For each cached text, the target text content is searched in other cached texts for overlapping content with the target text content;

[0263] When there is overlap in other cached texts, the length of the target text content excluding the overlap content is determined as the effective text length.

[0264] If there is no overlap with other cached texts, the length of the target text content is determined as the effective text length.

[0265] For cached text, if other cached texts contain overlapping content with the target text, then when evaluating the fluency of the cached text, the fluency evaluation results of the overlapping content can be reused. Therefore, only the text content outside the overlapping content in the target text needs to be evaluated for fluency. Thus, when overlapping content exists, the effective length can be the length of the text content in the target text excluding the overlapping content. When there is no overlapping content, the entire cached text needs to be evaluated for fluency, and the effective text length can be the length of the cached text itself.

[0266] Determining the effective text length of a cached text by identifying the intersection of its target text content with other cached texts can improve the accuracy of effective text length determination.

[0267] After determining the effective text length, multiple cached texts in the text fluency cache library can be filtered based on usage frequency, recent usage time, and effective text length.

[0268] In one implementation, filtering multiple cached texts in a text fluency cache library based on usage frequency, recent usage time, and effective text length includes:

[0269] The first computational cost is calculated based on usage frequency, the second computational cost is calculated based on the most recent usage time, and the third computational cost is calculated based on the effective text length.

[0270] The total computation cost of the corresponding cached text is determined based on the first computation cost, the second computation cost, and the third computation cost.

[0271] The text fluency cache library is filtered based on the total computational cost corresponding to each cached text.

[0272] The first computational cost can be measured from the perspective of usage frequency to determine the fluency evaluation cost of cached text. The higher the usage frequency of cached text, the greater the likelihood of it being reused. Therefore, after removing cached text from the text fluency cache library, it may be necessary to frequently perform repetitive fluency evaluations on text data matching the cached text, resulting in a high fluency evaluation cost. The first computational cost based on usage frequency can be calculated using a first preset function.

[0273] The second computational cost can be measured from the perspective of the most recent usage time (RUT). The closer the RUT is to the current time, the greater the likelihood that the RUT will be reused recently. Therefore, after removing RUT from the text fluency cache, the fluency of text data matching the RUT may need to be re-evaluated soon, resulting in a high fluency evaluation cost. The second computational cost based on RUT can be calculated using a second preset function.

[0274] The third computational cost can be measured from the perspective of the effective text length to assess the fluency of cached text. The longer the effective text length, the greater the computational cost required for fluency assessment, and therefore the higher the fluency assessment cost. The third computational cost based on the effective text length can be calculated using a third preset function.

[0275] In one implementation, the total computational cost of the cached text, determined based on the first computational cost, the second computational cost, and the third computational cost, can be calculated by summing them up, as shown in Formula 3:

[0276] c = g1(freq) + g2(time) + g3(len) (Formula 3).

[0277] In Formula 3, c can represent the total computation cost; freq can represent the frequency of use, g1(freq) can represent the first computation cost obtained by the first preset function g1 based on the frequency of use; time can represent the most recent use time, g2(time) can represent the second computation cost obtained by the second preset function g2 based on the most recent use time; len can represent the effective text length, g3(len) can represent the third computation cost obtained by the third preset function g3 based on the effective text length.

[0278] After calculating the total computational cost, multiple cached texts can be filtered based on the total computational cost corresponding to each cached text. During the filtering process, cached texts with higher total computational costs can be retained, while those with lower total computational costs can be removed. Specifically, cached texts with a total computational cost below a predetermined threshold can be removed from the text fluency cache library.

[0279] Filtering multiple cached texts based on the first, second, and third computational costs can retain cached texts that are difficult to evaluate and have a high probability of being reused, while removing cached texts that are easy to evaluate and have a low probability of being reused, thus improving the accuracy of text fluency cache library filtering.

[0280] In addition to usage frequency and recent usage time, cache filtering based on effective text length can retain cached text with high evaluation computational cost while removing cached text with low evaluation computational cost. While adjusting the memory space of the text fluency cache library, the efficiency of fluency evaluation based on the text fluency cache library is maintained.

[0281] Filtering multiple cached texts in the text fluency cache according to predetermined conditions can ensure that the cached texts in the text fluency cache can be continuously updated, and cached texts that are used less frequently or have not been used for a long time will be removed. This avoids the problem of the text fluency cache memory being occupied by cached texts with low reuse probability, which would lead to a decrease in comparison efficiency.

[0282] In summary, in the text fluency evaluation method of this disclosure embodiment, the cached data in the text fluency cache library can be continuously updated. For example... Figure 10 As shown, the process of changing cached text in the text fluency cache library includes: adding initial cached text data; building the cached text data index corresponding to the text fluency cache library; reusing cached text data; updating cached text data; and filtering cached text data.

[0283] In summary, the embodiments of this disclosure can, during text fluency evaluation, search for matching texts that have already undergone fluency evaluation in a text fluency cache, and revise the text fluency evaluation plan based on the matching texts. Thus, when executing the second text fluency evaluation plan, it is not necessary to start the fluency evaluation of the text data from the first word segment; instead, the fluency evaluation can be performed on the remaining text content excluding the matching text based on the matching text, and the second text fluency corresponding to the text data can be calculated using the matching fluency corresponding to the matching text and the first text fluency corresponding to the remaining text content. Therefore, the embodiments of this disclosure can fully reuse the fluency evaluation values ​​of some texts cached in the text fluency cache, thereby greatly reducing the computational load when evaluating the fluency of text data and improving the efficiency of text fluency evaluation.

[0284] This disclosure provides a detailed description of embodiments in conjunction with specific application scenarios.

[0285] like Figure 11 The diagram shown illustrates the specific process of applying the text fluency assessment method provided in this disclosure to a Chinese input scenario. The text fluency assessment method includes:

[0286] Step 1101: Obtain multiple matching texts based on the string input by the object, and obtain multiple candidate texts from the multiple matching texts.

[0287] The string input by the object can be the pinyin string entered by the object based on the desired Chinese input. Multiple matching texts can be found in the Chinese text library based on the string input by the object.

[0288] Due to the large number of matching texts, to improve Chinese input efficiency, multiple matching texts can be filtered based on predetermined filtering criteria to obtain multiple candidate texts. Preset filtering criteria could be: the top-ranked matching texts in terms of usage frequency within a predetermined time period. Usage frequency indicates how many times a matching text has been used within the predetermined time period and can be stored through text usage records. The higher the usage frequency of a matching text, the higher its probability of being selected. When filtering multiple matching texts, they can be sorted based on their usage frequency, and the top-ranked matching texts can be selected as candidate texts.

[0289] Step 1102: Generate multiple text data based on the associated context and multiple candidate texts.

[0290] The preceding text can be content that has already been entered into the input device and is related to the candidate text. The preceding text and the candidate text can form a coherent paragraph of text without punctuation. Concatenating the preceding text with each candidate text yields the text data corresponding to the candidate text.

[0291] When the string input by the object is preceded by a tag symbol or has no characters, and the associated previous text corresponding to the candidate text is empty, the text data can be the candidate text itself.

[0292] Step 1103: Obtain the text smoothness cache library, and query each target text data in the text smoothness cache library. When there is a smoothness evaluation value corresponding to the target text data in the text smoothness cache library, determine the smoothness evaluation value as the target smoothness evaluation value, and the smoothness evaluation of the target text data ends.

[0293] The text smoothness cache library contains multiple cached texts and the smoothness evaluation value corresponding to each cached text. The cached text can represent a text for which the smoothness has been calculated, and the smoothness evaluation value can represent the result obtained after calculating the smoothness of the cached text.

[0294] For each target text data among multiple text data, when the text smoothness cache library contains the target text data, the smoothness evaluation value corresponding to the target text data can be directly obtained as the target smoothness evaluation value corresponding to the candidate text corresponding to the target text data. At this time, the smoothness evaluation of the target text data ends, and the smoothness evaluation of the next target text data continues.

[0295] Step 1104: When there is no smoothness evaluation value corresponding to the target text data in the text smoothness cache library, for the target text data, perform word segmentation processing in the order of characters, and generate multiple consecutive texts based on the word segmentation processing result.

[0296] When there is no smoothness evaluation value corresponding to the target text data in the text smoothness cache library, word segmentation processing can be performed on the target text data to check whether the prefix data in the target text data is included in the text smoothness cache library.

[0297] Performing word segmentation processing on the target text data can segment the target text data by characters. For example, if the target text data is "Welcome", the word segmentation processing result is "Huan", "Ying", "Guang", and "Lin". Multiple consecutive texts are generated based on the word segmentation processing result, and the multiple consecutive texts contain different numbers of consecutive segments including the first segment. For example, the multiple consecutive texts generated based on the above word segmentation processing result can include: "Huan", "Welcome", "Welcome Guang", and "Welcome".

[0298] Step 1105: Search for the corresponding matching text in the text smoothness cache library one by one in the order of the number of characters contained in the multiple consecutive texts from less to more, and determine the target matching text and the text to be evaluated from the target text data.

[0299] The text fluency caching library can store multiple cached texts using a trie data structure. The trie stores multiple cached texts in character order as tree node paths. Therefore, matching texts are searched level by level in the trie corresponding to the text fluency caching library, in ascending order of the number of characters contained in multiple consecutive texts.

[0300] Therefore, for the first consecutive text, which contains only the first character of the target text data, the consecutive text is searched in the first level of the trie. If a match is successful, the matched text is obtained, and the second character of the second consecutive text is searched in the child node corresponding to the matched text in the trie, and so on. If a match fails, then matching subsequent consecutive text can be stopped. The last consecutive text that successfully matches is determined as the target matched text, and the text content in the target text data other than the target matched text is determined as the text to be evaluated.

[0301] Step 1106: If the target matching text does not exist in the text fluency cache, the target text data is identified as the text to be evaluated, and the process jumps to step 1109.

[0302] If no matching text exists for any consecutive text in the text fluency cache, it means that there is no cached text in the text fluency cache that can be reused for the target text data. Therefore, the target text data can be identified as the text to be evaluated, and the process can proceed to step 1109 for fluency evaluation.

[0303] Step 1107: Obtain the matching fluency and matching model state parameters corresponding to the target matching text.

[0304] The text fluency cache library can store multiple cached texts, along with the fluency evaluation value and model state parameters for each cached text. The fluency evaluation value is the result of evaluating the fluency of the cached text; the model state parameters can be the model state of a preset neural network model after calculating the fluency evaluation value for the cached text.

[0305] After determining the target matching text, you can retrieve the matching fluency and matching model state parameters corresponding to the target matching text.

[0306] Step 1108: Update the preset neural network model using the matching model state parameters.

[0307] The matching model's state parameters are passed into a preset neural network model, updating the preset neural network model's parameters. This allows the preset neural network model to perform fluency evaluation based on the fluency evaluation parameters of the target matching text when evaluating the text to be evaluated.

[0308] In this way, even if the target matching text in the target text data and the text to be evaluated are not evaluated for fluency simultaneously, when using the preset neural network model to evaluate the fluency of the text to be evaluated, the content of the two texts can be associated, thereby improving the accuracy of the fluency evaluation of the target text data.

[0309] Step 1109: Input the text to be evaluated into the preset neural network model to obtain the target fluency evaluation value.

[0310] The preset neural network model can evaluate the fluency of the text to be evaluated. Specifically, after inputting the text to be evaluated into the preset neural network model, the text to be evaluated can be tokenized first, and the preset neural network model is used to evaluate the coherence degree between each token and the content of the previous text, and then the target fluency evaluation value of the entire text to be evaluated can be obtained.

[0311] For example, if the text to be evaluated is "Welcome", its corresponding tokenization result is "Huan", "Ying", "Guang", and "Lin". The preset neural network model will first evaluate the coherence degree between "Huan" and the content of the previous text. Since there is no corresponding previous text for "Huan", the probability of "Huan" appearing when the previous text content is empty can be calculated; then, the coherence degree between "Ying" and the previous text content "Huan" is evaluated; then, the coherence degree between "Guang" and the previous text content "Welcome" is evaluated; finally, the coherence degree between "Lin" and "Welcome Guang" is evaluated. Finally, the coherence degree between each token and the corresponding previous text can be added up to obtain the target fluency evaluation value of the text to be evaluated.

[0312] Step 1110: Update the text fluency cache library.

[0313] After the preset neural network model evaluates the coherence degree between each token and the corresponding previous text content in the previous step, the coherence degree corresponding to the token can be added to the fluency evaluation value corresponding to the previous text content to obtain the fluency evaluation value of the text content composed of the previous text content and the token. For example, the fluency evaluation value corresponding to "Welcome" is equal to the sum of the coherence degree corresponding to "Huan" and the coherence degree corresponding to "Ying"; the fluency evaluation value corresponding to "Welcome Guang" is equal to the sum of the coherence degree corresponding to "Guang" and the fluency evaluation value of "Welcome".

[0314] In addition, after calculating the coherence degree between each token and the corresponding previous text content, the preset neural network model can retain the model state information of the preset neural network model as the model state parameter corresponding to the continuous text composed of the token and the corresponding previous text content.

[0315] For each continuous text in the target text data, if the continuous text is not included in the text fluency cache library, the continuous text can be used as a cached text, and it can be stored in the text fluency cache library together with the corresponding fluency evaluation value and the model state parameters.

[0316] In addition, the text fluency cache library also includes the usage frequency and the most recent usage time corresponding to the cached text. When there is a target matching text corresponding to the target text data in the text fluency cache library, the usage frequency corresponding to the target matching text is incremented by 1, and the most recent usage time is updated to the fluency evaluation time of the target text data. When adding a new cached text to the text fluency cache library, the usage frequency corresponding to the cached text is initialized to 1, and the most recent usage time is initialized to the new addition time of the cached text.

[0317] Step 1111: Sort multiple candidate texts based on the target fluency evaluation value corresponding to each candidate text for recommendation to the object.

[0318] After obtaining the target fluency evaluation value corresponding to each candidate text, the multiple candidate texts can be sorted based on the target fluency evaluation value, and the candidate texts can be recommended to the object according to the sorting result.

[0319] For example, the associated previous text is "Welcome", and the multiple candidate texts include "逛邻", "广林", "光临", "光林", and "广邻", and their corresponding target fluency evaluation values are: 0.8, 1.8, 2.5, 1.1, and 1.3 respectively. The result of recommending candidate texts to the object based on the target fluency evaluation value is: "光临", "广林", "广邻", "光林", and "逛邻".

[0320] In this way, the fluency evaluation method of the embodiments of the present disclosure can improve the efficiency of text fluency evaluation, and further improve the Chinese input efficiency of the object.

[0321] Description of the device and equipment of the embodiments of the present disclosure

[0322] It can be understood that although each step in the above various flowcharts is sequentially shown according to the indication of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless there is a clear indication in this embodiment, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowcharts may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0323] It should be noted that in various specific embodiments of this application, when processing is required based on data related to the characteristics of the target content, such as target content attribute information or attribute information sets, permission or consent from the target content provider will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require obtaining target content attribute information, separate permission or consent from the target content provider will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the separate permission or consent from the target content provider will the necessary target content-related data for the normal operation of the embodiments of this application be obtained.

[0324] Figure 12 A schematic diagram of the structure of a text fluency assessment device 1200 provided in an embodiment of this disclosure. The text fluency assessment device 1200 includes:

[0325] The first acquisition unit 1210 is used to acquire text data to be evaluated for fluency. The text data consists of candidate text corresponding to the string input by the input object and related preceding text.

[0326] The word segmentation unit 1220 is used to segment text data and generate a first text fluency evaluation plan based on the word segmentation results.

[0327] The second acquisition unit 1230 is used to acquire a text fluency cache library, which contains multiple cached texts and a fluency evaluation value corresponding to each cached text.

[0328] The correction unit 1240 is used to correct the first text fluency evaluation plan based on the comparison result between the cached text and the text data to obtain the second text fluency evaluation plan. The comparison result includes the matching text and the matching fluency corresponding to the matching text. The matching text is a part of the text in the text data.

[0329] The calculation unit 1250 is used to execute a second text fluency assessment plan and calculate the second text fluency corresponding to the text data based on the first text fluency and the matching fluency obtained from the execution of the second text fluency assessment plan.

[0330] Optionally, in one embodiment, the correction unit 1240 is specifically used for:

[0331] The text data is compared with the cached text in the text fluency cache library, and the target cached text is determined in the text fluency cache library based on the comparison results.

[0332] At least one redundant step is identified in the multiple first evaluation steps included in the first text fluency evaluation plan based on the target cached text;

[0333] The first text fluency assessment plan is modified according to at least one redundant step to obtain the second text fluency assessment plan.

[0334] Optionally, in one embodiment, the correction unit 1240 is specifically used for:

[0335] The multiple words determined by the word segmentation results are sorted according to the text order in the text data, and multiple first consecutive word groups containing the source word are generated based on the sorting order and multiple words. The source word is the word sorted first.

[0336] Multiple first consecutive phrases are compared with cached text in the text fluency cache library to obtain multiple matching second consecutive phrases;

[0337] The consecutive word group containing the most word segments among multiple second consecutive word groups is identified as the target cached text.

[0338] Optionally, in one embodiment, the correction unit 1240 is specifically used for:

[0339] Obtain the assessment objectives corresponding to each first assessment step in the first text fluency assessment plan. The assessment objectives include the assessment text and the assessment fluency value.

[0340] The first evaluation step that determines whether the evaluation text matches the target cached text is the matching step;

[0341] Based on the execution order among multiple first evaluation steps in the first text fluency evaluation plan, the preceding steps of the matching steps are determined, and at least one redundant step is determined based on the preceding steps and the matching steps.

[0342] Optionally, in one embodiment, the correction unit 1240 is specifically used for:

[0343] By eliminating the evaluation steps that are consistent with redundant steps in the first text fluency evaluation plan, multiple second evaluation steps are obtained;

[0344] The second text fluency assessment plan is generated based on the execution order of multiple second assessment steps and the steps in the first text fluency assessment plan.

[0345] Optionally, in one embodiment, the text fluency cache library further includes model state parameters corresponding to each cached text, and the model state parameters indicate the model state of the preset neural network model after calculating the fluency evaluation value corresponding to the cached text.

[0346] The computing unit 1250 is specifically used for:

[0347] Retrieve the matching model state parameters corresponding to the matched text from the text fluency cache library;

[0348] The preset neural network model is updated based on the matching model state parameters, and the second text fluency evaluation plan is executed using the updated preset neural network model.

[0349] Optionally, in one embodiment, the text fluency assessment device 1200 further includes:

[0350] The third acquisition unit (not shown) is used to determine at least one text to be stored based on the text data and the matching text, and to acquire the text to be stored fluency assessment value corresponding to each text to be stored based on the execution process of the second text fluency assessment plan.

[0351] An update unit (not shown) is used to update the text fluency cache based on the text to be stored and the corresponding fluency evaluation value to be stored.

[0352] Optionally, in one embodiment, the text fluency assessment device 1200 further includes:

[0353] The fourth acquisition unit (not shown) is used to acquire the usage frequency and recent usage time of each cached text in the text fluency cache when the text fluency cache reaches a predetermined condition.

[0354] A filtering unit (not shown) is used to filter multiple cached texts in the text fluency cache library based on usage frequency and recent usage time.

[0355] Alternatively, in one implementation, the filtering unit (not shown) is specifically used for:

[0356] Get the effective text length of each cached text;

[0357] The text fluency cache library is filtered based on usage frequency, recent usage time, and effective text length.

[0358] Alternatively, in one implementation, the filtering unit (not shown) is specifically used for:

[0359] For each cached text, the target text content is searched in other cached texts for overlapping content with the target text content;

[0360] When there is overlap in other cached texts, the length of the target text content excluding the overlap content is determined as the effective text length.

[0361] If there is no overlap with other cached texts, the length of the target text content is determined as the effective text length.

[0362] Alternatively, in one implementation, the filtering unit (not shown) is specifically used for:

[0363] The first computational cost is calculated based on usage frequency, the second computational cost is calculated based on the most recent usage time, and the third computational cost is calculated based on the effective text length.

[0364] The total computation cost of the corresponding cached text is determined based on the first computation cost, the second computation cost, and the third computation cost.

[0365] The text fluency cache library is filtered based on the total computational cost corresponding to each cached text.

[0366] Optionally, in one embodiment, the second acquisition unit 1230 is specifically used for:

[0367] Retrieve the text identifier corresponding to the text data;

[0368] The text fluency cache library corresponding to the text data is determined from multiple candidate text fluency cache libraries based on the text identifier.

[0369] Reference Figure 13 , Figure 13 To implement the structural block diagram of a portion of the terminal 140 according to an embodiment of this disclosure, the terminal includes: a radio frequency (RF) circuit 1310, a memory 1315, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (WiFi) module 1370, a processor 1380, and a power supply 1390, among other components. Those skilled in the art will understand that... Figure 13 The terminal 140 structure shown does not constitute a limitation on a mobile phone or computer, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0370] The RF circuit 1310 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 1380; in addition, it transmits uplink data to the base station.

[0371] The memory 1315 can be used to store software programs and modules. The processor 1380 executes various functional applications of the content terminal and text fluency assessments by running the software programs and modules stored in the memory 1315.

[0372] The input unit 1330 can be used to receive input numeric or character information, and to generate key signal inputs related to the settings and function control of the content terminal. Specifically, the input unit 1330 may include a touch panel 1331 and other input devices 1332.

[0373] The display unit 1340 can be used to display input or provided information, as well as various menus of the content terminal. The display unit 1340 may include a display panel 1341.

[0374] Audio circuitry 1360, speaker 1361, and microphone 1362 provide an audio interface.

[0375] In this embodiment, the processor 1380 included in the object terminal 140 can execute the text fluency evaluation method of the previous embodiment.

[0376] The target terminal 140 in this disclosure includes, but is not limited to, mobile phones, computers, intelligent voice exchange devices, smart home appliances, vehicle terminals, and aircraft. This invention can be applied to various scenarios, including but not limited to input methods and text detection.

[0377] Figure 14 This is a partial structural block diagram of a server 110 implementing an embodiment of the present disclosure. The server 110 can vary significantly due to different configurations or performance characteristics, and may include one or more central processing units (CPUs) 1422 (e.g., one or more processors) and memory 1432, and one or more storage media 1430 (e.g., one or more mass storage devices) for storing application programs 1442 or data 1444. The memory 1432 and storage media 1430 may be temporary or persistent storage. The program stored in the storage media 1430 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1422 may be configured to communicate with the storage media 1430 and execute the series of instruction operations stored in the storage media 1430 on the server.

[0378] Server 110 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input / output interfaces 1458, and / or one or more operating systems 1441, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0379] The central processing unit 1422 in server 110 can be used to execute the text fluency evaluation method of the present disclosure embodiments.

[0380] This disclosure also provides a computer-readable storage medium for storing program code for executing the text fluency evaluation methods of the foregoing embodiments.

[0381] This disclosure also provides a computer program product comprising a computer program. A processor of an electronic device reads and executes the computer program, causing the electronic device to perform the text fluency evaluation method described above.

[0382] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar terms and are not necessarily used to describe a particular order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0383] It should be understood that in this disclosure, "at least one item" refers to one or more items, and "more than one item" refers to two or more items. "And / or" is used to describe the relationship between related content, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related content are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0384] It should be understood that in the description of the embodiments disclosed herein, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0385] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0386] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0387] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0388] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0389] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0390] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0391] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A method for evaluating text fluency, characterized in that, include: Obtain the text data to be evaluated for fluency, which consists of candidate text corresponding to the string input by the input object and related preceding text; The text data is segmented into words, and a first text fluency assessment plan is generated based on the segmentation results; Obtain a text fluency cache library, which contains multiple cached texts and a fluency evaluation value for each cached text; The first text fluency evaluation plan is modified based on the comparison result between the cached text and the text data to obtain a second text fluency evaluation plan. The comparison result includes the matching text and the matching fluency corresponding to the matching text. The matching text is a part of the text in the text data. The second text fluency assessment plan is executed, and the second text fluency corresponding to the text data is calculated based on the first text fluency obtained from the second text fluency assessment plan and the matching fluency.

2. The method according to claim 1, characterized in that, The step of revising the first text fluency evaluation plan based on the comparison result between the cached text and the text data to obtain a second text fluency evaluation plan includes: The text data is compared with the cached text in the text fluency cache library, and the target cached text is determined in the text fluency cache library based on the comparison result; Based on the target cached text, at least one redundant step is identified among the multiple first evaluation steps included in the first text fluency evaluation plan; The first text fluency assessment plan is modified according to the at least one redundant step to obtain the second text fluency assessment plan.

3. The method according to claim 2, characterized in that, The step of comparing the text data with cached text in the text fluency cache library, and determining the target cached text in the text fluency cache library based on the comparison result, includes: The multiple words determined by the word segmentation result are sorted according to the text order in the text data, and multiple first continuous word groups containing the source word are generated based on the sorting order and the multiple words, wherein the source word is the word sorted first. The plurality of first consecutive word groups are compared with the cached text in the text fluency cache library to obtain a plurality of matching second consecutive word groups; The consecutive word group containing the most word segments among the plurality of second consecutive word groups is identified as the target cached text.

4. The method according to claim 2, characterized in that, The step of determining at least one redundant step in the plurality of first evaluation steps included in the first text fluency evaluation plan based on the target cached text includes: Obtain the evaluation target corresponding to each first evaluation step in the first text fluency evaluation plan, wherein the evaluation target includes the evaluation text and the evaluation fluency value; The first evaluation step that determines whether the evaluation text matches the target cached text is the matching step; The preceding steps of the matching step are determined based on the execution order among the multiple first evaluation steps in the first text fluency evaluation plan, and at least one redundant step is determined based on the preceding steps and the matching steps.

5. The method according to claim 4, characterized in that, The step of revising the first text fluency assessment plan according to the at least one redundant step to obtain a second text fluency assessment plan includes: By removing the evaluation steps in the first text fluency evaluation plan that are consistent with the redundant steps, multiple second evaluation steps are obtained; The second text fluency assessment plan is generated based on the execution order of the multiple second assessment steps and the steps in the first text fluency assessment plan.

6. The method according to any one of claims 1-5, characterized in that, The text fluency cache library also includes model state parameters corresponding to each cached text. The model state parameters indicate the model state of the preset neural network model after calculating the fluency evaluation value corresponding to the cached text. The execution of the second text fluency assessment plan includes: Retrieve the matching model state parameters corresponding to the matching text from the text fluency cache library; The preset neural network model is updated based on the matching model state parameters, and the second text fluency evaluation plan is executed using the updated preset neural network model.

7. The method according to any one of claims 1-6, characterized in that, After executing the second text fluency assessment plan and calculating the second text fluency corresponding to the text data based on the first text fluency assessed by executing the second text fluency assessment plan and the matching fluency, the method further includes: Based on the text data and the matching text, at least one text to be stored is determined, and based on the execution process of the second text fluency assessment plan, a fluency assessment value corresponding to each text to be stored is obtained. The text fluency cache is updated based on the text to be stored and the corresponding fluency evaluation value to be stored.

8. The method according to any one of claims 1-6, characterized in that, The method further includes: When the text fluency cache library meets the predetermined conditions, the usage frequency and most recent usage time of each cached text in the text fluency cache library are obtained; The multiple cached texts in the text fluency cache library are filtered based on the usage frequency and the most recent usage time.

9. The method according to claim 8, characterized in that, The filtering of multiple cached texts in the text fluency cache library based on the usage frequency and the most recent usage time includes: Obtain the effective text length for each cached text; The multiple cached texts in the text fluency cache are filtered based on the usage frequency, the most recent usage time, and the effective text length.

10. The method according to claim 9, characterized in that, The step of obtaining the effective text length of each cached text includes: For each cached text target text content, query other cached texts for overlapping content with the target text content; When the other cached texts contain the intersection content, the length of the target text content excluding the intersection content is determined as the effective text length. If the intersection content does not exist in the other cached texts, the length of the target text content is determined as the effective text length.

11. The method according to claim 9, characterized in that, The filtering of multiple cached texts in the text fluency cache library based on the usage frequency, the most recent usage time, and the effective text length includes: A first computational cost is calculated based on the frequency of use, a second computational cost is calculated based on the most recent usage time, and a third computational cost is calculated based on the effective text length. The total computation cost of the corresponding cached text is determined based on the first computation cost, the second computation cost, and the third computation cost. The multiple cached texts in the text fluency cache library are filtered based on the total computational cost corresponding to each cached text.

12. The method according to any one of claims 1-11, characterized in that, The text fluency caching library includes: Obtain the text identifier corresponding to the text data; Based on the text identifier, determine the text fluency cache library corresponding to the text data from multiple candidate text fluency cache libraries.

13. A text fluency assessment device, characterized in that, include: The first acquisition unit is used to acquire text data to be evaluated for fluency, wherein the text data consists of candidate text corresponding to the string input by the input object and related preceding text; The word segmentation unit is used to perform word segmentation processing on the text data and generate a first text fluency evaluation plan based on the word segmentation results; The second acquisition unit is used to acquire a text fluency cache library, which contains multiple cached texts and a fluency evaluation value corresponding to each cached text. The correction unit is used to correct the first text fluency evaluation plan based on the comparison result between the cached text and the text data to obtain a second text fluency evaluation plan. The comparison result includes the matching text and the matching fluency corresponding to the matching text. The matching text is a part of the text in the text data. The calculation unit is configured to execute the second text fluency assessment plan and calculate the second text fluency corresponding to the text data based on the first text fluency assessed by executing the second text fluency assessment plan and the matching fluency.

14. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the text fluency evaluation method according to any one of claims 1 to 12.

15. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the text fluency assessment method according to any one of claims 1 to 12.

16. A computer program product comprising a computer program that is read and executed by a processor of an electronic device, causing the electronic device to perform the text fluency assessment method according to any one of claims 1 to 12.

Images