An article text processing method, device and equipment

Abstract

Embodiments of the present specification disclose an article text processing method, device and equipment. The scheme comprises: obtaining an automatically generated article text; splitting the article text by sentences to obtain a plurality of corresponding sentence texts; determining one or more first detection models obtained according to a large language model and a second detection model obtained according to a named entity recognition model; detecting the sentence texts and the article text by using the first detection model to determine whether there is a quality problem, wherein the quality problem detected for the sentence texts is at least partially different from the quality problem detected for the sentence texts; if it is determined that the sentence texts have a quality problem, the second detection model is used to continue detecting the sentence texts with the quality problem to determine the specific position where the quality problem occurs; and outputting the obtained detection result.

Description

Technical Field

[0001] This specification relates to the field of text processing technology, and in particular to a method, apparatus, and device for text processing. Background Technology

[0002] With the rapid development of internet technology and the widespread use of smart terminals, many traditional businesses can now be conducted online using smart terminal applications. Moreover, many new businesses have emerged.

[0003] For example, there's the business of automatically generating open-source text, such as automatically generating story text, including fairy tales and online novels. It's important to note that automatic open-source text generation is a crucial but highly challenging task in natural language generation. Here, "open-source" refers to inputs containing only a small amount of information needed to generate the target output, such as the story's title and brief content. In contrast, tasks like machine translation and text summarization fall under the category of non-open-source generation.

[0004] Currently, in practical applications, the quality of automatically generated article texts, such as story text automatic generation, varies greatly. When these article texts are provided as service content, it may affect the user experience and lead to unnecessary waste of resources, including both service processing resources and users' time and money.

[0005] Therefore, there is a need for solutions that can improve the quality of text generated in the automatic text generation business of open-source articles, so as to improve user experience and avoid resource waste. Summary of the Invention

[0006] This specification provides one or more embodiments of an article text processing method, apparatus, device, and storage medium to solve the following technical problem: the need for a solution that helps improve the quality of generated text in open-source article text automatic generation services, so as to improve user experience and avoid resource waste.

[0007] To solve the above-mentioned technical problems, one or more embodiments of this specification are implemented as follows:

[0008] This specification provides one or more embodiments of an article text processing method, including:

[0009] Get the automatically generated article text;

[0010] The article text is split into sentences to obtain multiple corresponding sentence texts;

[0011] Determine one or more first detection models obtained based on a large language model, and a second detection model obtained based on a named entity recognition model;

[0012] The first detection model is used to detect the sentence text and the article text to determine whether there are quality problems, wherein the quality problems detected for the sentence text are at least partially different from the quality problems detected for the sentence text.

[0013] If it is determined that the sentence text has a quality problem, then the second detection model is used to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs;

[0014] Output the obtained detection results.

[0015] Optionally, the step of using the first detection model to detect the sentence text and the article text to determine whether there are quality problems specifically includes:

[0016] Using at least one of the one or more first detection models, each of the sentence texts is detected to determine whether a predefined quality problem exists in the sentence text;

[0017] By using at least one of the one or more first detection models, the entire text of the article is directly detected, or the text of the article is detected at the paragraph level, to determine whether there are predefined quality problems in the entire text of the article, and the details of the existing quality problems.

[0018] Optionally, the step of detecting each sentence text using at least one of the one or more first detection models specifically includes:

[0019] Using at least one of the one or more first detection models, detect whether each sentence text has at least one of the following predefined quality problems:

[0020] Poor guidance, factual errors, inconsistent protagonists, inappropriate word choice, and illogical reasoning.

[0021] Optionally, the step of directly detecting the entire article text or detecting the article text at the paragraph level using at least one of the one or more first detection models specifically includes:

[0022] Using at least one of the one or more first detection models, the entire article text can be directly detected, or the article text can be detected at the paragraph level to determine whether it has at least one of the following predefined quality problems:

[0023] The plot is repetitive, the story goes off-topic, and the logic is flawed.

[0024] Optionally, before outputting the obtained detection result, the method further includes:

[0025] Determine the third detection model obtained from the large language model;

[0026] The third detection model is used to detect the article text to determine whether the article text possesses at least one of the following predefined quality advantages:

[0027] The plot is engaging and educational.

[0028] Optionally, the article text is story text.

[0029] This specification provides one or more embodiments of an article text processing apparatus, comprising:

[0030] The article text acquisition module retrieves the automatically generated article text.

[0031] The article sentence segmentation module splits the article text into sentences, resulting in multiple corresponding sentence texts.

[0032] The detection model determination module determines one or more first detection models obtained based on the large language model, and a second detection model obtained based on the named entity recognition model.

[0033] Multiple granularity detection modules utilize the first detection model to detect the sentence text and the article text to determine whether quality problems exist, wherein the quality problems detected for the sentence text are at least partially different from the quality problems detected for the sentence text.

[0034] If the problem location detection module determines that the sentence text has a quality problem, it uses the second detection model to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs.

[0035] The detection result output module outputs the obtained detection results.

[0036] This specification provides one or more embodiments of an article text processing device, comprising:

[0037] At least one processor; and,

[0038] A memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0040] Get the automatically generated article text;

[0041] The article text is split into sentences to obtain multiple corresponding sentence texts;

[0042] Determine one or more first detection models obtained based on a large language model, and a second detection model obtained based on a named entity recognition model;

[0043] The first detection model is used to detect the sentence text and the article text to determine whether there are quality problems, wherein the quality problems detected for the sentence text are at least partially different from the quality problems detected for the sentence text.

[0044] If it is determined that the sentence text has a quality problem, then the second detection model is used to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs;

[0045] Output the obtained detection results.

[0046] This specification provides one or more embodiments of a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0047] Get the automatically generated article text;

[0048] The article text is split into sentences to obtain multiple corresponding sentence texts;

[0049] Determine one or more first detection models obtained based on a large language model, and a second detection model obtained based on a named entity recognition model;

[0050] The first detection model is used to detect the sentence text and the article text to determine whether there are quality problems, wherein the quality problems detected for the sentence text are at least partially different from the quality problems detected for the sentence text.

[0051] If it is determined that the sentence text has a quality problem, then the second detection model is used to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs;

[0052] Output the obtained detection results.

[0053] The above-mentioned at least one technical solution adopted in one or more embodiments of this specification can achieve the following beneficial effects: it can divide the automatically generated article text on the open end into different granularities, including sentence granularity and coarser granularity such as the whole article or paragraphs. Different quality problems to be detected are defined according to the characteristics of different granularities. Based on at least part of the capabilities of the large language model, quality problems are detected for different granularities of the article text. If quality problems are found, the specific location of the quality problem can be further located based on at least part of the capabilities of the named entity recognition model, thereby providing reliable, accurate, specific and interpretable quality detection results. The quality detection results help improve the quality of the automatically generated article text on the open end, avoid providing users with poor-quality article text, and avoid wasting resources for service providers and users. Moreover, it also provides different quality problems suitable for different detection granularities for the application scenario of automatically generated story text on the open end, thereby automatically generating excellent story text that is more suitable for users, especially minors. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating an article text processing method provided in one or more embodiments of this specification;

[0056] Figure 2 A schematic diagram of a player interface for playing automatically generated short stories, provided for one or more embodiments of this specification;

[0057] Figure 3 A flowchart illustrating a quality control scheme for an automated story text generation business scenario provided in one or more embodiments of this specification;

[0058] Figure 4 This diagram illustrates the detectable quality issues and advantages of automatically generated article text in a practical application scenario provided by one or more embodiments of this specification, defining story text for automatic article text.

[0059] Figure 5 In a practical application scenario provided by one or more embodiments of this specification, how to specifically address different granularities... Figure 1 A flowchart illustrating the process for detecting these quality issues in the proposed solution;

[0060] Figure 6 This document provides a flowchart illustrating the process of generating a first detection model based on a large language model for a business scenario of automatic generation of story text, as provided in one or more embodiments of this specification.

[0061] Figure 7 In the business scenarios of automatic story text generation provided by one or more embodiments of this specification, Figure 1 A flowchart illustrating one implementation scheme of the method;

[0062] Figure 8 A schematic diagram of the structure of an article text processing device provided in one or more embodiments of this specification;

[0063] Figure 9 This is a schematic diagram of the structure of an article text processing device provided for one or more embodiments of this specification. Detailed Implementation

[0064] This specification provides an article text processing method, apparatus, device, and storage medium through its embodiments.

[0065] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0066] In response to the issues raised in the background technology, we will further analyze the issue using the automatic generation of story text on open-source platforms as an example.

[0067] Storybooks, as a medium for entertainment, education, and intellectual enlightenment, have become an important part of the culture of countries around the world. Research on open-source story text generation is of great significance. First, exploring the intrinsic mechanisms of text content creation can help realize human-like artificial intelligence that masters language intelligence; second, open-source long text generation also has important application value in entertainment, education, and human-computer interaction.

[0068] In their actual work, the applicant specifically encountered a business scenario involving the automatic generation of long fairy tales (e.g., 800-1000 words) on open platforms. To ensure the quality of the generated long fairy tales and avoid the problems of low efficiency and high cost caused by manual quality inspection, it is urgent to implement quality inspection of the generated long fairy tale text results on open platforms based on a large model.

[0069] The applicant discovered that, faced with the complex requirements of quality inspection for generated long fairy tale texts on open-source platforms, some of their previous attempts at non-open-source text generation quality inspection solutions were not highly applicable, failing to yield reliable inspection results at different granularities, and also lacking a comprehensive system of detectable quality issues suitable for this scenario. To address these issues, the applicant not only provided a quality inspection solution for automatically generated article texts on open-source platforms but also further proposed a system of detectable quality issues more suitable for the long fairy tale text generation scenario, to complement this quality inspection solution.

[0070] Based on this overall concept, the scheme of this application will be further described in detail below.

[0071] Figure 1 This is a flowchart illustrating an article text processing method provided in one or more embodiments of this specification. The process can be executed by devices such as the application's front-end (e.g., smartphone, tablet, etc.) or back-end (e.g., application server, etc.). Intermediate results involved in the process (e.g., sentence segmentation results, preliminary detection results of the existence of quality issues, quality issue location information, etc.) can also be manually intervened to improve the accuracy of the final quality detection results.

[0072] Figure 1 The process includes the following steps:

[0073] S102: Get the automatically generated article text.

[0074] In one or more embodiments of this specification, the article text can be pre-generated by the current executing entity or other entities. For example, a short text prompting the content generation target can be input to the corresponding entity in advance. The entity receives the short text, processes it through a specified generation model, and automatically generates an article text longer than or even much longer than the short text according to the content generation target. The automatic generation of open-source article text mentioned above generally conforms to this example.

[0075] In this context, the short text can be user-provided or machine-provided information such as qualifying words, phrases, or short sentences that provide a general description of the content of the article to be generated, with the expectation that the automatically generated article should conform to these general descriptions as much as possible. The generated article, however, is much more specific than the short text. For example, the short text might only have a few, a dozen, or even a few dozen words, while the generated article might have hundreds, thousands, or even more words.

[0076] Figure 2This diagram illustrates a player interface for playing automatically generated short stories, provided for one or more embodiments of this specification. Some content is obscured in the diagram but does not affect the understanding of the solution presented in this application. Figure 2 In this scenario, the story text for the short story is automatically generated and further converted into audio for playback through the player. The page contains two information tags: "Short Story Title" and "Story Synopsis." In one application scenario, users can input information under these tags as short text prompts for content generation. The application containing the player can then automatically generate the story text based on the user-inputted prompts in the "Short Story Title" and "Story Synopsis."

[0077] For example, if the user input prompt includes "Generate a story about a stray cat and a squirrel", the generated story text may include "A beautiful stray cat often stayed alone in a corner. He looked very lonely because no owner took care of him. One day, he met a squirrel standing on a skateboard in the park. He was very curious and ran around the stray cat, looking very interested..."

[0078] S104: Split the article text into sentences to obtain multiple corresponding sentence texts.

[0079] In one or more embodiments of this specification, each sentence text corresponds to a sentence. Only a portion of the sentence text in the article text can be selected for detection. For example, sentences with sufficiently complex content or sufficiently long length can be identified and detected.

[0080] S106: Determine one or more first detection models obtained from a large language model, and a second detection model obtained from a named entity recognition (NER) model.

[0081] In one or more embodiments of this specification, when multiple first detection models are available, these multiple first detection models may employ the same or similar model structures, but be trained using different training samples, so as to be used for different specific detection objects. Large language models, for example, may employ ChatGLM-6B, LLaMa-ch, baichuan-7b, or other models with similar capabilities.

[0082] The first detection model can directly use a large language model itself, or a model modified based on a large language model or its principles. The second detection model can directly use a named entity recognition model itself, or a model modified based on a named entity recognition model or its principles.

[0083] S108: Detect the sentence text and the article text using the first detection model to determine whether there is a quality problem, wherein the quality problem detected for the sentence text is at least partially different from the quality problem detected for the sentence text.

[0084] In one or more embodiments of this specification, in order to improve the targeting, sensitivity and reliability of detection for different quality problems, multiple different first detection models are pre-trained. The first detection model used to detect sentence text is not the same model as the first detection model used to detect article text.

[0085] This application provides, in particular, a detectable quality problem system more suitable for story text generation scenarios, which defines multiple quality problems.

[0086] For example, quality issues such as misleading, factual errors, inconsistent protagonists, repetitive plots, and off-topic stories are defined. In the context of fairy tale text generation, where the generated text is likely intended for minors, the issue of misleading is particularly noteworthy. Defining quality issues can include defining corresponding model classification labels, judgment thresholds, and training sample standards to facilitate model training and inference against specific quality issues.

[0087] The detection granularity for each quality problem can be preset, which helps to achieve relatively higher detection accuracy for the corresponding quality problem at different granularities.

[0088] In one or more embodiments of this specification, two or more granularities are defined. One is sentence granularity (referred to as fine-grained granularity), which is reflected by detecting sentence text, and is relatively more suitable for detecting some subtle, well-defined, and less controversial quality issues. The others are coarser granularities than sentence granularity (referred to as coarse-grained granularity), such as overall article granularity, paragraph granularity, etc., which are reflected by detecting article text, and are relatively more suitable for detecting some quality issues that hinder the overall user experience, have more flexible boundaries, and are more controversial.

[0089] For story texts, especially fairy tales, fine-grained quality checks may affect minors' objective cognition (e.g., knowledge and values), while coarse-grained quality checks may affect minors' interest in the story text. Therefore, the multi-granularity classification and corresponding quality issues adapted to the specific granularity, as described in this application, are particularly advantageous in the automatic generation of story texts on open platforms, better adapting to the characteristics of this business compared to other types of text. Taking non-open platform text translation as an example, this type of business primarily focuses on the accuracy of mapping between different languages, without considering quality requirements such as interest and objective cognition.

[0090] S110: If it is determined that the sentence text has a quality problem, then the second detection model is used to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs.

[0091] In one or more embodiments of this specification, the first detection model can be used to roughly determine that there is a quality problem in the article text or a specific sentence text, and then the sentence text or article text with quality problem can be further detected to determine the specific location where the quality problem occurs.

[0092] However, this application has further considerations, including: for coarse-grained article texts or paragraph texts, the content is usually much more extensive than that of individual sentences. In this case, if the second detection model is also used to detect these coarse-grained contents with quality problems, it may lead to two adverse effects: First, the resources consumed by detection will increase significantly, and the efficiency will decrease significantly; second, for article texts or paragraph texts with more holistic quality problems, it may be inappropriate to locate them in overly fragmented local locations. Furthermore, the one or more local locations located (which may not be able to locate all of them) may not be sufficient to completely cause the holistic quality problem. As a result, the specific location of the detected quality problem may have a negative effect and be misleading, which is not conducive to accurately and precisely understanding and correcting the holistic quality problem.

[0093] Based on this consideration, the first detection model detects each sentence individually. If a sentence is found to have a predefined quality problem, the first detection model outputs a concise detection result for that sentence. The second detection model then continues to detect the sentence to pinpoint the specific location of the quality problem (e.g., a particular word), which is then used as the second detection model's output. Alternatively, the first detection model performs a coarser-grained detection on the entire article or paragraphs. If a predefined quality problem is detected in the entire article or paragraphs, the first detection model outputs a more detailed detection result (achieved through differentiated training of different first detection models). However, the second detection model is not used to further detect the entire article or paragraphs. In other words, when the first detection model detects a quality problem in the entire article, compared to detecting a quality problem in a single sentence, the first detection model provides a more detailed output (e.g., a more holistic description and analysis of the content corresponding to the quality problem within the entire article, rather than just locating a single word). This avoids the two adverse effects mentioned above and helps achieve the corresponding beneficial effects.

[0094] S112: Output the obtained detection results.

[0095] In one or more embodiments of this specification, the detection results output by the first detection model and / or the detection results output by the second detection model are output.

[0096] Furthermore, based on the detection results, it is possible to control whether the generated article text is provided to users. Additionally, it can be used to guide the optimization of automatically generated article text. Intuitively, one or more embodiments of this specification provide a flowchart illustrating a quality control scheme in a business scenario of automatic story text generation. See [link to documentation]. Figure 3 .

[0097] exist Figure 3 The process comprises two parts: the left side controls whether the automatically generated story text is provided to the user, and can be executed online; the right side controls whether the automatic story text generation model is deployed online, and can be executed offline. Both parts utilize the aforementioned scheme for detecting quality issues in the automatically generated article text.

[0098] The process on the left includes the following steps: Obtaining a pre-trained story text generation model; users who need story text providing feedback on their specific needs (inputting a small amount of information to the model) to indicate what kind of story text they require; the story text generation model then automatically generates the story text based on this feedback; and finally, utilizing... Figure 1The proposed solution involves performing a quality check on the story text to determine if it meets the requirements. If not, the text is temporarily withheld from the user to avoid impacting the user experience, and the story text can be regenerated using a story text generation model. If the text meets the requirements, the current story text is then displayed to the user for their use.

[0099] The process on the right includes the following steps: training and iterating the story text generation model using the training sample dataset; automatically generating story text using the story text generation model to check whether the model currently meets expectations; and then using... Figure 1 The proposed solution involves performing quality checks on these story texts to determine whether the story text generation model currently meets expectations. If not, the model is further trained and iterated to optimize it. If it does meet expectations, the story text generation model is then deployed for more formal use in business.

[0100] pass Figure 1 This method can segment automatically generated article text from open platforms at different granularities, including sentence-level and coarser granularities such as the entire article or paragraphs. Different quality issues to be detected are defined to suit the characteristics of different granularities. Based on at least part of the capabilities of a large language model, quality issues are detected at different granularities of the article text. If quality issues are found, the method can further pinpoint the specific location of the quality issues based on at least part of the capabilities of a named entity recognition model. This provides reliable, accurate, specific, and highly interpretable quality detection results. These results help improve the quality of automatically generated article text from open platforms, avoiding providing users with substandard text and preventing waste of resources for both service providers and users. Furthermore, specifically for the application scenario of automatically generated story text from open platforms, it provides different quality issues suitable for different detection granularities, thereby automatically generating excellent story text more suitable for users, especially minors.

[0101] based on Figure 1 In addition to the method described herein, this specification also provides some specific implementation schemes and extension schemes of this method, which will be further explained below.

[0102] In one or more embodiments of this specification, in addition to detecting quality problems, it is also possible to detect the existence of quality advantages. This allows for a more objective and complete evaluation of the quality of automatically generated article text and helps to efficiently select better article text. In particular, when users have a high tolerance for quality problems, quality advantages can receive more attention from users.

[0103] Based on this, before outputting the obtained detection results, a third detection model obtained from the large language model can be determined. The third detection model is then used to detect the article text to determine whether the article text has predefined quality advantages, and the details of the quality advantages can also be analyzed.

[0104] To further illustrate the quality advantages and quality issues, one or more embodiments of this specification also provide a schematic diagram of detectable quality issues and quality advantages for automatically generated article text in a practical application scenario, referring to [reference needed]. Figure 4 These quality issues and advantages are particularly well-suited for business scenarios involving the automatic generation of fairy tale texts.

[0105] exist Figure 4 In the quality assessment, there are two positive aspects, which are considered bonus points: "Educational" and "Attractive Plot." There are seven negative aspects, which are considered deductions: Level 1 (unacceptable): "Poor guidance" and "Factual errors"; Level 2 (not entirely unacceptable): "Off-topic story" and "Inconsistent protagonist"; Level 3 (a relatively high percentage of not entirely unacceptable): "Logical inconsistencies" and "Repetitive plot"; and Level 4 (a relatively minor): "Inappropriate word choice."

[0106] Based on these exemplary quality issues, one or more embodiments of this specification also provide a practical application scenario, at different granularities, specifically how to... Figure 1 See the flowchart illustrating the process for detecting these quality issues in the proposed solution. Figure 5 .

[0107] S502: Using at least one of one or more first detection models obtained from a large language model, detect whether each of the sentence texts has at least one of the following predefined quality problems: poor guidance, factual errors, inconsistent protagonists, inappropriate word choice, and illogical reasoning.

[0108] S504: Using at least one of one or more first detection models obtained from a large language model, directly detect the entire text of the article, or detect at the paragraph level whether the text of the article has at least one of the following predefined quality problems: plot repetition, story digression, or logical incoherence.

[0109] Here, the quality assessment task for fairy tale texts is broken down into two main granularities: fine-grained (sentence-level) and coarse-grained (article / paragraph-level). Quality issues such as poor guidance, factual errors, inconsistent protagonists, and inappropriate word choice, which are considered deduction points, will be assessed at the sentence level because these issues tend to be localized. Conversely, issues like plot repetition and digressions will be assessed at the article / paragraph level because these issues tend to be broader or more widespread. Logical inconsistencies, due to their conceptual specificity, require both fine-grained and coarse-grained assessment.

[0110] S506: Using a third detection model obtained from a large language model; using the third detection model to detect the article text to determine whether the article text has at least one of the following predefined quality advantages: engaging plot, educational value.

[0111] For fairy tales aimed at minors, it is especially important to focus on detecting quality issues such as inappropriate guidance, as well as quality advantages such as being clean and engaging.

[0112] It should be noted that the definitions and classifications of these quality issues and advantages, as well as the established correspondences between them and different particle size detection processes, are exemplary. In practical applications, they can be readjusted according to the ideas and principles analyzed above.

[0113] In one or more embodiments of this specification, to improve detection efficiency and better facilitate online applications, a first detection model is obtained by specifically simplifying the large language model. Specifically, one or more embodiments of this specification provide a flowchart illustrating the generation of the first detection model based on the large language model for a business scenario involving automatic generation of story text. (See attached diagram.) Figure 6 .

[0114] Figure 6 The process includes the following steps:

[0115] S602: Before determining the first detection model obtained based on the large language model, inference and training are performed on the specified large language model using the story sample text and the non-story sample text.

[0116] In the specific context of fairy tale samples, non-fairy tale samples can be used for non-story sample texts. This makes it easier to be more sensitive to story texts aimed at minors after subsequent training, and helps to conduct more accurate quality testing from the perspective of protecting minors.

[0117] S604: Based on the inference results of the training iterations, detect the difference in sensitivity of the parameters in the specified large language model to the non-story sample text compared to the story sample text.

[0118] S606: Obtain at least some parameters from the set of parameters with sufficiently high sensitivity differences.

[0119] It should be noted that when computing power is low or efficiency requirements are high, it is not necessary to be so selective in obtaining these parameters. Other methods can be considered to obtain at least some parameters of a given large language model more easily, such as random selection or selecting parameters that change relatively little during model training iterations, etc.

[0120] S608: By converting at least some of the parameters to low precision and saving them again, a simplified model corresponding to the large language model is generated as the first detection model.

[0121] The first detection model generated in this way can improve detection efficiency while preserving as much of the sensitivity that a large language model can have to story sample text as possible.

[0122] Similarly, one can first determine a set of parameters with sufficiently low sensitivity, and then obtain some parameters from outside this set, convert them to low precision, and save them again.

[0123] Based on the above description, one or more embodiments of this specification also provide a business scenario for automatic story text generation. Figure 1 A flowchart of one implementation scheme of the method is shown below. Figure 7 .

[0124] The scheme consists of two parallel parts: sentence-level quality detection and overall / paragraph quality detection. For example, the ChatGLM-6B model is used as the first and third detection models, and the NER model is used as the second detection model.

[0125] Taking fairy tale texts as an example. In the sentence-level quality detection section, for an input fairy tale text, it is first split according to the sentence dimension. Then, the split sentences are input into a pre-trained ChatGLM-6B model to coarsely determine whether the current input sentence has corresponding quality problems. For sentences with corresponding quality problems, a NER model is used to further determine the specific location of the problem, such as word subscripts. In the overall / paragraph quality detection section, for an input fairy tale text, it is directly input into another pre-trained ChatGLM-6B model, which can output more details of the corresponding problems.

[0126] Take the overall / paragraph quality assessment section of the article as an example.

[0127] Suppose that the fairy tale text was automatically generated with the theme "A stray cat became friends with a squirrel who could skateboard in the park". The specific content of the generated fairy tale text is omitted here.

[0128] After inputting the fairy tale text into the corresponding ChatGLM-6B model for detection, the following detection results might be output:

[0129] The details such as "the stray cat and the kitten," "the stray cat and the skateboarding squirrel," "ultimately, the stray cat and the skateboarding squirrel decided to leave the pet store," "moved together into the forest," and "the skateboarding squirrel told the stray cat that he lived in a nearby forest and came to skateboard every day. The stray cat was happy to hear this and decided to go to the forest to see the skateboarding squirrel" all reflect a story that is off-topic and illogical.

[0130] Take the sentence granularity quality detection section as an example.

[0131] Suppose we generate a story sentence text, "The little fish's father begins to doubt the little fish's authenticity," and the corresponding ChatGLM-6B model detects it, it might output the following detection results:

[0132] The word "authenticity" is used inappropriately in this sentence.

[0133] The solution provided in this application offers a more comprehensive range of text quality detection dimensions for stories (especially fairy tales), which helps to evaluate the quality of story generation more comprehensively, accurately and specifically. Moreover, the detection results are highly interpretable, able to specifically locate the problem and provide details of the problem. The model used can also support longer texts and has good support for Chinese texts, thus improving the applicability of the solution.

[0134] Based on the same idea, one or more embodiments of this specification also provide apparatus and devices corresponding to the above methods, such as... Figure 8 , Figure 9 As shown. The apparatus and equipment are capable of performing the above methods and related alternatives accordingly.

[0135] Figure 8 This specification provides a schematic diagram of the structure of an article text processing device according to one or more embodiments, the device comprising:

[0136] Article text acquisition module 802 acquires automatically generated article text;

[0137] The article sentence segmentation module 804 splits the article text into sentences to obtain multiple corresponding sentence texts;

[0138] The detection model determination module 806 determines one or more first detection models obtained based on the large language model, and a second detection model obtained based on the named entity recognition model;

[0139] The multi-granularity detection module 808 uses the first detection model to detect the sentence text and the article text to determine whether there are quality problems, wherein the quality problems detected for the sentence text are at least partially different from the quality problems detected for the sentence text.

[0140] If the problem location detection module 810 determines that the sentence text has a quality problem, it uses the second detection model to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs.

[0141] The detection result output module 812 outputs the obtained detection results.

[0142] Optionally, it also includes:

[0143] The article text generation module 814 receives short text to prompt the target of the generated content before obtaining the automatically generated article text;

[0144] The short text is processed by a specified generation model to automatically generate an article text that is much longer than the short text.

[0145] Optionally, the detection model determination module 806 acquires at least some parameters of the specified large language model before determining the first detection model obtained based on the large language model;

[0146] By converting at least some of the parameters to low precision and saving them again, a simplified model corresponding to the large language model is generated as the first detection model.

[0147] Optionally, the detection model determination module 806 uses the story sample text and the non-story sample text to perform inference and training in a specified large language model;

[0148] Based on the inference results of the training iterations, detect the difference in sensitivity of the parameters in the specified large language model to the non-story sample text compared to the story sample text;

[0149] Obtain at least some parameters from the set of parameters with sufficiently high sensitivity differences.

[0150] Optionally, the multi-granularity detection module 808 uses at least one of the one or more first detection models to detect each sentence text respectively, so as to determine whether there is a predefined quality problem in the sentence text;

[0151] By using at least one of the one or more first detection models, the entire text of the article is directly detected, or the text of the article is detected at the paragraph level, to determine whether there are predefined quality problems in the entire text of the article, and the details of the existing quality problems.

[0152] Optionally, after the multi-granularity detection module 808 uses the first detection model to detect the sentence text and the article text to determine whether there is a quality problem, when the first detection model detects a quality problem in the article text, it obtains a detection result with relatively more detailed content output by the first detection model compared to when the first detection model detects a quality problem in a single sentence text.

[0153] Optionally, the multiple granularity detection module 808 uses at least one of the one or more first detection models to detect whether each sentence text has at least one of the following predefined quality problems:

[0154] Poor guidance, factual errors, inconsistent protagonists, inappropriate word choice, and illogical reasoning.

[0155] Optionally, the multi-granularity detection module 808 uses at least one of the one or more first detection models to directly detect the entire article text, or to detect whether the article text has at least one of the following predefined quality problems at the paragraph level:

[0156] The plot is repetitive, the story goes off-topic, and the logic is flawed.

[0157] Optionally, before outputting the obtained detection results, the detection model determination module 806 determines a third detection model obtained based on the large language model;

[0158] The multi-granularity detection module 808 uses the third detection model to detect the article text to determine whether the article text has at least one of the following predefined quality advantages:

[0159] The plot is engaging and educational.

[0160] Optionally, the article text is story text.

[0161] Figure 9This specification provides a schematic diagram of the structure of an article text processing device according to one or more embodiments, the device comprising:

[0162] At least one processor; and,

[0163] A memory communicatively connected to the at least one processor; wherein,

[0164] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0165] Get the automatically generated article text;

[0166] The article text is split into sentences to obtain multiple corresponding sentence texts;

[0167] Determine one or more first detection models obtained based on a large language model, and a second detection model obtained based on a named entity recognition model;

[0168] The first detection model is used to detect the sentence text and the article text to determine whether there are quality problems, wherein the quality problems detected for the sentence text are at least partially different from the quality problems detected for the sentence text.

[0169] If it is determined that the sentence text has a quality problem, then the second detection model is used to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs;

[0170] Output the obtained detection results.

[0171] Based on the same idea, one or more embodiments of this specification also provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0172] Get the automatically generated article text;

[0173] The article text is split into sentences to obtain multiple corresponding sentence texts;

[0174] Determine one or more first detection models obtained based on a large language model, and a second detection model obtained based on a named entity recognition model;

[0175] The first detection model is used to detect the sentence text and the article text to determine whether there are quality problems, wherein the quality problems detected for the sentence text are at least partially different from the quality problems detected for the sentence text.

[0176] If it is determined that the sentence text has a quality problem, then the second detection model is used to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs;

[0177] Output the obtained detection results.

[0178] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0179] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0180] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0181] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0182] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0183] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0184] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0185] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0186] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0187] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0188] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A text processing method for articles, comprising: Get the automatically generated article text; The article text is split into sentences to obtain multiple corresponding sentence texts; Determine one or more first detection models obtained based on a large language model, and a second detection model obtained based on a named entity recognition model; The first detection model is used to detect the sentence text and the article text to determine whether there are quality problems, wherein the quality problems detected for the sentence text are at least partially different from the quality problems detected for the sentence text. If it is determined that the sentence text has a quality problem, then the second detection model is used to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs; Output the obtained detection results.

2. The method as described in claim 1, wherein before obtaining the automatically generated article text, the method further includes: Receive short texts to prompt the target of content generation; The short text is processed by a specified generation model to automatically generate an article text that is much longer than the short text.

3. The method of claim 1, wherein before determining the first detection model obtained based on the large language model, the method further comprises: Retrieves at least some parameters of a specified large language model; By converting at least some of the parameters to low precision and saving them again, a simplified model corresponding to the large language model is generated as the first detection model.

4. The method as described in claim 3, wherein obtaining at least some parameters of the specified large language model specifically includes: Inference and training are performed on a specified large language model using story sample text and non-story sample text. Based on the inference results of the training iterations, detect the difference in sensitivity of the parameters in the specified large language model to the non-story sample text compared to the story sample text; Obtain at least some parameters from the set of parameters with sufficiently high sensitivity differences.

5. The method as described in claim 1, wherein the step of using the first detection model to detect the sentence text and the article text to determine whether a quality problem exists specifically includes: Using at least one of the one or more first detection models, each of the sentence texts is detected to determine whether a predefined quality problem exists in the sentence text; By using at least one of the one or more first detection models, the entire text of the article is directly detected, or the text of the article is detected at the paragraph level, to determine whether there are predefined quality problems in the entire text of the article, and the details of the existing quality problems.

6. The method of claim 5, wherein after using the first detection model to detect the sentence text and the article text to determine whether a quality problem exists, the method further includes: When the first detection model detects quality issues in the article text, compared to when the first detection model detects quality issues in a single sentence text, the detection model outputs a detection result with relatively more detailed content.

7. The method of claim 5, wherein detecting each sentence text using at least one of the one or more first detection models specifically includes: Using at least one of the one or more first detection models, detect whether each sentence text has at least one of the following predefined quality problems: Poor guidance, factual errors, inconsistent protagonists, inappropriate word choice, and illogical reasoning.

8. The method as described in claim 5, wherein the step of directly detecting the entire text of the article or detecting the text of the article at the paragraph level using at least one of the one or more first detection models specifically includes: Using at least one of the one or more first detection models, the entire article text can be directly detected, or the article text can be detected at the paragraph level to determine whether it has at least one of the following predefined quality problems: The plot is repetitive, the story goes off-topic, and the logic is flawed.

9. The method of claim 5, wherein before outputting the obtained detection result, the method further comprises: Determine the third detection model obtained from the large language model; The third detection model is used to detect the article text to determine whether the article text possesses at least one of the following predefined quality advantages: The plot is engaging and educational.

10. The method according to any one of claims 1 to 9, wherein the article text is story text.

11. An article text processing device, comprising: The article text acquisition module retrieves the automatically generated article text. The article sentence segmentation module splits the article text into sentences, resulting in multiple corresponding sentence texts. The detection model determination module determines one or more first detection models obtained based on the large language model, and a second detection model obtained based on the named entity recognition model. Multiple granularity detection modules utilize the first detection model to detect the sentence text and the article text to determine whether quality problems exist, wherein the quality problems detected for the sentence text are at least partially different from the quality problems detected for the sentence text. If the problem location detection module determines that the sentence text has a quality problem, it uses the second detection model to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs. The detection result output module outputs the obtained detection results.

12. The apparatus of claim 11, further comprising: The article text generation module receives short text that prompts the target content of the generated content before obtaining the automatically generated article text; The short text is processed by a specified generation model to automatically generate an article text that is much longer than the short text.

13. The apparatus of claim 11, wherein the detection model determination module acquires at least some parameters of a specified large language model before determining the first detection model obtained based on the large language model; By converting at least some of the parameters to low precision and saving them again, a simplified model corresponding to the large language model is generated as the first detection model.

14. The apparatus of claim 13, wherein the detection model determination module performs reasoning and training within a specified large language model using story sample text and non-story sample text; Based on the inference results of the training iterations, detect the difference in sensitivity of the parameters in the specified large language model to the non-story sample text compared to the story sample text; Obtain at least some parameters from the set of parameters with sufficiently high sensitivity differences.

15. The apparatus of claim 11, wherein the multiple granularity detection module uses at least one of the one or more first detection models to detect each of the sentence texts respectively, so as to determine whether there is a predefined quality problem in the sentence text; By using at least one of the one or more first detection models, the entire text of the article is directly detected, or the text of the article is detected at the paragraph level, to determine whether there are predefined quality problems in the entire text of the article, and the details of the existing quality problems.

16. The apparatus of claim 15, wherein the multi-granularity detection module, after detecting the sentence text and the article text using the first detection model to determine whether there is a quality problem, obtains a detection result with relatively more detailed content output by the first detection model when the first detection model detects a quality problem in the article text, compared to when the first detection model detects a quality problem in a single sentence text.

17. The apparatus of claim 15, wherein the multiple granularity detection modules, using at least one of the one or more first detection models, respectively detect whether each sentence text has at least one of the following predefined quality problems: Poor guidance, factual errors, inconsistent protagonists, inappropriate word choice, and illogical reasoning.

18. The apparatus of claim 15, wherein the multiple granularity detection module uses at least one of the one or more first detection models to directly detect the entire text of the article, or to detect whether the text of the article has at least one of the following predefined quality problems at the paragraph granularity level: The plot is repetitive, the story goes off-topic, and the logic is flawed.

19. The apparatus of claim 15, wherein before outputting the obtained detection result, the detection model determination module determines a third detection model obtained based on the large language model; The multiple granularity detection modules utilize the third detection model to detect the article text to determine whether the article text possesses at least one of the following predefined quality advantages: The plot is engaging and educational.

20. The apparatus according to any one of claims 11 to 19, wherein the text is a story text.

21. An article text processing device, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform: Get the automatically generated article text; The article text is split into sentences to obtain multiple corresponding sentence texts; Determine one or more first detection models obtained based on a large language model, and a second detection model obtained based on a named entity recognition model; The first detection model is used to detect the sentence text and the article text to determine whether there are quality problems, wherein the quality problems detected for the sentence text are at least partially different from the quality problems detected for the sentence text. If it is determined that the sentence text has a quality problem, then the second detection model is used to continue detecting the sentence text with the quality problem in order to determine the specific location where the quality problem occurs; Output the obtained detection results.

Images