Text quality evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides a text quality evaluation method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the fields of large models, model generation, deep learning, text processing and the like. According to the specific implementation scheme, whether a to-be-evaluated text meets a preset condition or not is determined according to at least one of the total character number, the content category, the repetition rate and characters related to a preset language of the to-be-evaluated text; determining an adjustment evaluation value of the at least one evaluation parameter according to the processing logic corresponding to the at least one evaluation parameter under the condition of determining that the to-be-evaluated text meets the predetermined condition; and determining a quality evaluation result of the to-be-evaluated text according to the reference evaluation value and the adjustment evaluation value of the at least one evaluation parameter.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the fields of large models, generative models, deep learning, text processing, etc. More specifically, the present disclosure provides a text quality assessment method, device, electronic device, storage medium, and computer program product. Background Art

[0002] Large language models or other artificial intelligence models sometimes need to be trained using text. In practical applications, text screening and review are required to improve the quality of the text and thus ensure the training effect of the model. Manual review of text quality can be used, but this method is costly. Summary of the invention

[0003] The present disclosure provides a text quality assessment method, device, electronic device, storage medium and computer program product.

[0004] According to one aspect of the present disclosure, a text quality assessment method is provided, including: determining whether a text to be assessed meets a predetermined condition based on at least one of the total number of characters, content category, repetition rate, and characters related to a predetermined language of the text to be assessed; when it is determined that the text to be assessed meets the predetermined condition, determining an adjusted assessment value of at least one assessment parameter based on a processing logic corresponding to at least one assessment parameter; and determining a quality assessment result of the text to be assessed based on a respective baseline assessment value and an adjusted assessment value of at least one assessment parameter.

[0005] According to another aspect of the present disclosure, a text quality assessment device is provided, comprising: a first determination module, a second determination module, and a third determination module. The first determination module is used to determine whether the text to be assessed meets a predetermined condition based on at least one of the total number of characters, content category, repetition rate, and characters related to a predetermined language of the text to be assessed; the second determination module is used to determine an adjusted assessment value of at least one assessment parameter according to a processing logic corresponding to at least one assessment parameter when it is determined that the text to be assessed meets the predetermined condition; the third determination module is used to determine a quality assessment result of the text to be assessed based on the respective baseline assessment value and the adjusted assessment value of at least one assessment parameter.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided by the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method provided by the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided by the present disclosure is implemented.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram of an application scenario of the text quality assessment method and device according to an embodiment of the present disclosure;

[0012] Figure 2 is a schematic flow chart of a text quality assessment method according to an embodiment of the present disclosure;

[0013] Figure 3 is a schematic diagram of a text quality assessment method according to an embodiment of the present disclosure;

[0014] Figure 4 is a schematic structural block diagram of a text quality assessment device according to an embodiment of the present disclosure; and

[0015] Figure 5 It is a structural block diagram of an electronic device used to implement the text quality assessment method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0017] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0018] In the technical solution of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0019] In some embodiments, the quality of the text can be evaluated by manual screening and manual review, and then the high-quality text can be determined from a large amount of text. However, this method has the following disadvantages, which are mainly reflected in the following aspects.

[0020] First, the labor cost is high. With the development of large model training and natural language processing technology, the amount of data required has increased dramatically. Relying on a large number of manual processes will lead to a sharp increase in costs and low efficiency.

[0021] Second, it is highly subjective. Manual evaluation is easily affected by personal experience, understanding ability, and subjective judgment, resulting in inconsistency and bias in the evaluation results. This subjectivity may reduce the accuracy and reliability of the evaluation.

[0022] Third, poor scalability. Faced with large-scale data sets, the speed of manual evaluation is far behind the speed of data growth and it is difficult to meet real-time requirements. In addition, with the advancement of technology and the emergence of new application scenarios, evaluation standards and requirements will continue to change, and manual review will find it difficult to adapt to these changes in a timely manner.

[0023] Fourth, the error rate is high. Although manual evaluation can guarantee quality to a certain extent, errors such as missed detection and misjudgment may still occur due to factors such as fatigue and negligence. These errors may have a negative impact on subsequent model training and application.

[0024] Fifth, it is difficult to continuously optimize. The manual evaluation process is relatively fixed, and it is difficult to quickly optimize and adjust based on feedback results. However, automated evaluation tools such as machine learning can continuously optimize the evaluation model by continuously learning user feedback, thereby improving the accuracy and efficiency of the evaluation.

[0025] The present disclosure aims to provide a text quality assessment method, which first determines whether the text to be assessed meets predetermined conditions based on at least one of the total number of characters, content category, repetition rate, and characters related to a predetermined language of the text to be assessed, thereby preliminarily screening the text. If the predetermined conditions are met, the adjusted evaluation value of the evaluation parameter is determined, and then the quality assessment result of the text to be assessed is determined based on the baseline evaluation value and the adjusted evaluation value of the evaluation parameter. The assessed text with high quality can be used for the corpus of a large model. The text quality assessment method provided by the present disclosure can efficiently screen high-quality texts, improve evaluation efficiency and accuracy, and reduce labor costs.

[0026] The text quality assessment method provided by the present disclosure can be applied to fields such as large model training, natural language processing, artificial intelligence writing assistance, scientific research and education, and enterprise document management.

[0027] The technical solution provided by the present disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] Figure 1 Schematic diagram of an application scenario of a text quality assessment method and device according to an embodiment of the present disclosure.

[0029] It should be noted that Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0030] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, a server 105, and a database 106. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0031] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptops, desktop computers, etc.

[0032] The database 106 may store a plurality of texts to be evaluated, and may also store evaluation results of the texts.

[0033] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results to the terminal device.

[0034] For example, the server 105 can obtain the text to be evaluated from the terminal devices 101, 102, 103, or extract the text to be evaluated from the database 106, and then evaluate the text to be evaluated to obtain the evaluation result. The server 105 can also return the evaluation result to the terminal devices 101, 102, 103 to let the user know the evaluation result, and can also store the evaluation result in the database 106. In addition, the server 105 can train the artificial intelligence model with the high-quality text after evaluation according to actual needs.

[0035] It should be noted that the text quality assessment method provided in the embodiment of the present disclosure can generally be performed by the server 105. Accordingly, the text quality assessment device provided in the embodiment of the present disclosure can generally be set in the server 105. The text quality assessment method provided in the embodiment of the present disclosure can also be performed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the text quality assessment device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0036] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0037] Figure 2 is a schematic flow chart of a text quality assessment method according to an embodiment of the present disclosure.

[0038] like Figure 2 As shown, the text quality assessment method 200 may include operations S210 to S240.

[0039] In operation S210, it is determined whether the text to be evaluated meets a predetermined condition according to at least one of the total number of characters, the content category, the repetition rate, and characters related to a predetermined language of the text to be evaluated.

[0040] For example, the text to be evaluated may be text in a document, and the document may include documents of Word, PPT, PDF, Excel, etc. For example, the text to be evaluated includes multiple attributes.

[0041] For example, an attribute is the total number of characters, which may include the length of the parsed text string, and may include punctuation marks, numbers, special symbols, and the like in addition to regular characters such as Chinese characters and English characters.

[0042] For example, another attribute is the content category of the text to be evaluated, which may include sports, animation, travel, etc. The content category can be detected by a pre-trained classification model.

[0043] For example, another attribute is the repetition rate of the text to be evaluated, which can indicate the degree of repetition of the text compared with other texts in the database.

[0044] For example, another attribute is character information of characters related to a predetermined language. The predetermined language may include Chinese, English, etc. Taking Chinese as an example, the characters related to the predetermined language refer to Chinese characters, and the character information may include information such as the number and proportion of characters related to the predetermined language.

[0045] For example, corresponding reference ranges may be pre-set for attributes such as the total number of characters, content category, repetition rate, character information of characters related to a predetermined language, and if the value of at least one attribute exceeds the value range, it is determined that the text to be evaluated does not meet the predetermined conditions; otherwise, it is determined that the predetermined conditions are met.

[0046] In operation S220 , an adjusted evaluation value of at least one evaluation parameter is determined according to a processing logic corresponding to the at least one evaluation parameter.

[0047] For example, if the evaluation parameter is the abnormal character ratio, the processing logic may be to determine the ratio between the number of abnormal characters and the total number of characters. The corresponding relationship between the ratio and the adjusted evaluation value may be preconfigured, for example, if the ratio is 0.1, the adjusted evaluation value is -1.

[0048] In operation S230, a quality evaluation result of the to-be-evaluated text is determined according to the respective baseline evaluation value and the adjusted evaluation value of at least one evaluation parameter.

[0049] For example, the corresponding relationship between the evaluation parameter and the benchmark evaluation value can be preconfigured. For example, the benchmark evaluation value and the adjusted evaluation value of the same evaluation parameter can be added, weighted, or otherwise calculated to obtain a sub-evaluation value of the evaluation parameter. Then, the sub-evaluation values ​​of each evaluation parameter can be added, weighted, or otherwise calculated to obtain a total evaluation value, and the quality evaluation result can include the total evaluation value.

[0050] The technical solution provided by the embodiment of the present disclosure adjusts the evaluation value by attribute calculation, and then automatically evaluates it in combination with the basic evaluation value, thereby achieving the effect of automatically evaluating the text quality. Compared with the manual review method, the technical solution provided by the embodiment of the present disclosure can reduce costs and improve efficiency.

[0051] In addition, the technical solution provided by the embodiment of the present disclosure first preliminarily screens the text to be evaluated, and further determines and adjusts the evaluation value when the predetermined conditions are met. This method can first determine whether the quality of the text to be evaluated is too low based on important attributes, avoid the waste of resources caused by subsequent processing of the text with too low quality, and further improve the evaluation efficiency.

[0052] According to another embodiment of the present disclosure, the predetermined condition includes multiple sub-conditions. Next, each sub-condition will be described.

[0053] In this example, a sub-condition includes: the total number of characters in the text to be evaluated is not less than a predetermined number threshold. For example, the characters may include the length of the parsed text string, and in addition to regular characters such as Chinese characters and English characters, they may also include punctuation marks, numbers, special symbols, etc. The predetermined number threshold is 200, which is not limited in this embodiment. A small total number of characters in the text to be evaluated will result in less semantic information, which is not suitable for training the model, so this sub-condition is used to filter out this type of text.

[0054] In this example, another sub-condition includes: the content category of the text to be evaluated is different from the predetermined abnormal category. The category of the text to be evaluated can be determined by a pre-trained classification model. The predetermined abnormal category includes, for example, sensitive information. Using text containing such information to train the model will result in low model security. Therefore, it is not appropriate to use this sub-condition to filter out such text. The predetermined abnormal category can be flexibly selected according to actual needs, and this embodiment does not limit it.

[0055] In this example, another sub-condition includes: the repetition rate of the text to be evaluated is not greater than a predetermined repetition rate threshold. For example, the repetition rate threshold may be 85%, which is not limited in this embodiment.

[0056] In this example, another sub-condition includes: the ratio between the number of characters related to the predetermined language and the total number of characters in the text to be evaluated is not less than a first predetermined ratio. For example, the predetermined language may be Chinese, and the first predetermined ratio may be 0.7. This embodiment does not limit the predetermined language and the first predetermined ratio. A small ratio will result in less semantic information related to the predetermined language, which is not suitable for training the model, so this sub-condition is used to filter out this type of text.

[0057] In practical applications, the text to be evaluated can be determined to meet the predetermined condition when one or part of the sub-conditions are met, or it can be determined to meet the predetermined condition when all the sub-conditions are met.

[0058] According to another embodiment of the present disclosure, in the case where the text to be evaluated includes characters in a table, the predetermined condition may further include the following sub-condition: in the first target number of rows in the table, the ratio between the number of columns containing characters in a predetermined language and the total number of columns is not less than a second predetermined ratio. For example, the first target number of rows may be the first few rows, the last few rows, or a few randomly selected rows in the table, the first target number may be 10, and the second predetermined ratio may be, for example, 0.8. A ratio less than the second predetermined ratio indicates that the proportion of characters in the predetermined language in the table is too small, and this type of text is not suitable for training the model, so this type of text may be filtered out by this sub-condition.

[0059] According to another embodiment of the present disclosure, in the case where the text to be evaluated includes characters in a table, the predetermined condition may further include the following sub-condition: in the second target number column of the table, the ratio between the number of rows containing characters in a predetermined language and the total number of rows is not less than a third predetermined ratio. The second target number columns may be the first few columns, the last few columns, or a few randomly selected columns in the table, the second target number may be 10, and the second predetermined ratio may be, for example, 0.8. A ratio less than the second predetermined ratio indicates that the proportion of characters in the predetermined language in the table is too small, and this type of text is not suitable for training the model, so this type of text may be filtered out by this sub-condition.

[0060] According to another embodiment of the present disclosure, the process of determining the adjustment evaluation of at least one evaluation parameter according to the processing logic corresponding to the at least one evaluation parameter may include: processing the text to be evaluated based on the processing logic corresponding to the evaluation parameter to obtain a processing result. Then, according to the numerical range and mapping relationship of the processing result, the adjusted evaluation value of the evaluation parameter is determined, wherein the mapping relationship represents the relationship between the numerical range and the adjusted evaluation value, and the mapping relationship is preconfigured. Next, the evaluation parameters, processing logic, and the process of determining the adjusted evaluation value are described.

[0061] In one example, the evaluation parameter includes the proportion of abnormal characters, and the processing logic corresponding to the evaluation parameter is to determine the ratio of predetermined abnormal characters to the total number of characters. Abnormal characters can be pre-configured and stored in a dictionary, and the characters in the text can be compared with the characters in the dictionary, and determined to be abnormal characters when they are consistent. For example, abnormal characters may include the following characters: ×^↓↑§. For example, the numerical range includes 0~10%, 10%~20%, 20%~40%, 40~100%, and the adjusted evaluation values ​​corresponding to the above four ranges are 0, -5, -10, and -20, respectively. Taking the ratio of 5% as an example, the adjusted evaluation value is -5.

[0062] In another example, the evaluation parameter includes a word repetition rate, and the processing logic corresponding to the evaluation parameter is to determine the ratio between the word that appears at least twice and the total number of words after deduplication. For example, the numerical range includes 0~80%, 80%~85%, 85%~95%, 95~100%, and the adjusted evaluation values ​​corresponding to the above four ranges are 0, -5, -15, and -20, respectively. Taking the ratio of 5% as an example, the adjusted evaluation value is -5. For example, the text is "I love learning", in which "I" appears 2 times and the rest appear 1 time, so there is one repeated word and four words after deduplication, so the word repetition rate is 25%, and the word repetition rate is in the numerical range of 0~80%, so the adjusted evaluation values ​​are 0, respectively.

[0063] In another example, the evaluation parameter includes a word repetition rate, and the processing logic corresponding to the evaluation parameter is to determine the ratio between words that appear at least twice and the total number of words after deduplication. For example, some word segmentation tools can be used to segment the text to obtain multiple word segments in the text, and then the number of times each word appears is determined, and the word repetition rate is calculated. For example, the numerical range includes 0~75%, 75%~85%, 85%~95%, 95~100%, and the adjusted evaluation values ​​corresponding to the above four ranges are 0, -5, -15, and -20, respectively.

[0064] In another example, the evaluation parameter includes language perplexity (PPL), which characterizes the fluency of the text to be evaluated. The processing logic corresponding to the evaluation parameter is to determine the value of the language perplexity, and a pre-trained model can be used to determine the language perplexity. For example, the numerical range includes 0~500, 500~600, 600~700, 700~800, 800~900, 900~1000, 1000~1200, 1200~1400, and above 1400. The adjusted evaluation values ​​corresponding to the above nine ranges are 0, -2, -4, -6, -8, -12, -20, -30, and -40, respectively.

[0065] It should be noted that the evaluation parameters may include any one or more combinations of the above-mentioned abnormal character ratio, character repetition rate, word repetition rate and language perplexity.

[0066] In this embodiment, each evaluation parameter corresponds to a processing logic, based on which the processing result can be determined, and a corresponding adjustment evaluation value is configured for the numerical range of the processing result, so that the processing result can be accurately quantified, thereby improving the accuracy of the quality evaluation result.

[0067] According to another embodiment of the present disclosure, the process of determining the quality evaluation result of the text to be evaluated based on the respective baseline evaluation value and the adjusted evaluation value of at least one evaluation parameter may include: for each evaluation parameter of the at least one evaluation parameter, determining the sub-evaluation value of the evaluation parameter based on the baseline evaluation value and the adjusted evaluation value of the evaluation parameter, and then determining the quality evaluation result based on at least one sub-evaluation value of the at least one evaluation parameter.

[0068] For example, the baseline evaluation value and the adjusted evaluation value of the same evaluation parameter can be added to obtain the sub-evaluation value of the evaluation parameter. For example, if there are two evaluation parameters, the baseline evaluation value of one evaluation parameter is 40, and the adjusted evaluation value is -10, then the sub-evaluation value of the evaluation parameter is 30. In addition, if the baseline evaluation value of another evaluation parameter is 60, and the adjusted evaluation value is -5, then the sub-evaluation value of the evaluation parameter is 55.

[0069] For example, the benchmark assessment value of the above abnormal character ratio is 40, the benchmark assessment value of the character repetition rate is 20, the benchmark assessment value of the word repetition rate is 20, and the benchmark assessment value of the language perplexity is 20.

[0070] In one example, the quality assessment result may include a total assessment value. The sub-assessment values ​​of each assessment parameter may be added together, and the result obtained is used as the total assessment value. For example, if the sub-assessment value of one assessment parameter is 30 and the sub-assessment value of another assessment parameter is 55, then the total assessment value of the two assessment parameters is 85. This example determines the sub-assessment value of each assessment parameter, and then determines the total assessment value based on the assessment values ​​of multiple assessment parameters, so that the text to be assessed can be assessed from multiple assessment dimensions, thereby improving the accuracy of the quality assessment result.

[0071] In another example, the quality assessment result may include a quality grade. The correspondence between the total assessment value and the quality grade can be pre-configured, for example, if the total assessment value is not less than the first value, the quality grade is the first grade, if the total assessment value is less than the first value and not less than the second value, the quality grade is the second grade, if the total assessment value is less than the second value and not less than the third value, the quality grade is the third grade, and if the total assessment value is less than the third value, the quality grade is the fourth grade. The first value may be 94, the second value may be 84, and the third value may be 68, which is not limited in this embodiment. The text to be assessed can be divided into at least two grades by the quality grade, so that in the subsequent training model process, the text in the appropriate quality grade can be selected according to actual needs.

[0072] In practical applications, after obtaining the quality assessment results of the text, the text with higher quality can be used as the prediction for the training model, for example, text with a total assessment value above 85, or text with the first or second quality level.

[0073] Figure 3 It is a schematic diagram of a text quality assessment method according to an embodiment of the present disclosure.

[0074] According to another embodiment of the present disclosure, multiple documents 301 may be obtained, for example, documents 301 of the types of Word, PPT, PDF, Excel, etc., and then the text in the document 301 may be preprocessed, and the preprocessing may include parsing, filtering, etc. Next, the text in the preprocessed document 301 is used as the text to be evaluated 302, and the text quality evaluation method described above is used to evaluate the text to be evaluated 302 300, thereby obtaining a quality evaluation result 303.

[0075] In addition, several texts may be extracted from the multiple texts after evaluation 300 for testing to determine whether the quality evaluation result 303 is accurate. If accurate, texts with higher quality may be selected from the texts after evaluation and used to perform the model training task.

[0076] Figure 4 It is a schematic structural block diagram of a text quality assessment device according to an embodiment of the present disclosure.

[0077] like Figure 4 As shown, the text quality assessment apparatus 400 may include a first determination module 410 , a second determination module 420 and a third determination module 430 .

[0078] The first determination module 410 is used to determine whether the text to be evaluated meets a predetermined condition according to at least one of the total number of characters, content category, repetition rate, and characters related to a predetermined language of the text to be evaluated.

[0079] The second determination module 420 is used to determine an adjusted evaluation value of at least one evaluation parameter according to a processing logic corresponding to at least one evaluation parameter when it is determined that the to-be-evaluated text satisfies a predetermined condition.

[0080] The third determination module 430 is used to determine the quality evaluation result of the to-be-evaluated text according to the respective baseline evaluation value and the adjusted evaluation value of at least one evaluation parameter.

[0081] According to another embodiment of the present disclosure, the second determination module includes: a processing submodule and an adjusted evaluation value determination submodule. The processing submodule is used to process the text to be evaluated based on the processing logic corresponding to the evaluation parameter to obtain a processing result. The adjusted evaluation value determination submodule is used to determine the adjusted evaluation value of the evaluation parameter according to the numerical range of the processing result and the mapping relationship between the numerical range and the adjusted evaluation value.

[0082] According to another embodiment of the present disclosure, the third determination module includes: a sub-evaluation value determination sub-module and a result determination sub-module. The sub-evaluation value determination sub-module is used to determine the sub-evaluation value of the evaluation parameter according to the baseline evaluation value and the adjustment evaluation value of the evaluation parameter for each evaluation parameter in at least one evaluation parameter. The result determination sub-module is used to determine the quality evaluation result according to at least one sub-evaluation value of at least one evaluation parameter.

[0083] According to another embodiment of the present disclosure, the result determination submodule includes: a total evaluation value determination unit and a grade determination unit. The total evaluation value determination unit is used to determine the total evaluation value according to at least one sub-evaluation value of at least one evaluation parameter; the grade determination unit is used to determine the quality grade of the text to be evaluated according to the evaluation value range in which the total evaluation value is located; the quality evaluation result includes the quality grade.

[0084] According to another embodiment of the present disclosure, at least one evaluation parameter includes at least one of the following: abnormal character ratio, character repetition rate, word repetition rate, and language confusion. The abnormal character ratio represents the ratio of predetermined abnormal characters to the total number of characters; the character repetition rate represents the ratio between characters that appear at least twice and the total number of words after deduplication; the word repetition rate represents the ratio between words that appear at least twice and the total number of words after deduplication; and the language confusion represents the fluency of the text to be evaluated.

[0085] According to another embodiment of the present disclosure, the predetermined conditions include: the total number of characters in the text to be evaluated is not less than a predetermined number threshold; the content category of the text to be evaluated is different from a predetermined abnormal category; the repetition rate of the text to be evaluated is not greater than a predetermined repetition rate threshold; and the ratio between the number of characters related to the predetermined language and the total number of characters in the text to be evaluated is not less than a first predetermined ratio.

[0086] According to another embodiment of the present disclosure, in the case where the text to be evaluated includes a table, the predetermined condition also includes at least one of the following: in a target number of rows in the table, the ratio between the number of columns containing characters of a predetermined language and the total number of columns is not less than a second predetermined ratio; and in a target number of columns in the table, the ratio between the number of rows containing characters of a predetermined language and the total number of rows is not less than a third predetermined ratio.

[0087] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned text quality assessment method.

[0088] According to an embodiment of the present disclosure, the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the above-mentioned text quality assessment method.

[0089] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, including a computer program, and the computer program implements the above-mentioned text quality assessment method when executed by a processor.

[0090] Figure 5is a block diagram of an electronic device for implementing the text quality assessment method of an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0091] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0092] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0093] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the text quality assessment method. For example, in some embodiments, the text quality assessment method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the text quality assessment method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the text quality assessment method in any other appropriate manner (e.g., by means of firmware).

[0094] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0095] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0096] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0097] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0098] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0099] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0100] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0101] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A text quality assessment method, comprising: Determining whether the text to be evaluated meets a predetermined condition according to at least one of the total number of characters, content category, repetition rate, and characters related to a predetermined language in the text to be evaluated; In the case where it is determined that the text to be evaluated meets the predetermined condition, determining an adjusted evaluation value of the at least one evaluation parameter according to a processing logic corresponding to the at least one evaluation parameter; as well as A quality evaluation result of the to-be-evaluated text is determined according to the respective baseline evaluation value and the adjusted evaluation value of the at least one evaluation parameter.

2. The method according to claim 1, wherein: Determining the adjusted evaluation value of the at least one evaluation parameter according to the processing logic corresponding to the at least one evaluation parameter comprises: Processing the text to be evaluated based on the processing logic corresponding to the evaluation parameters to obtain a processing result; and The adjusted evaluation value of the evaluation parameter is determined according to the numerical range of the processing result and the mapping relationship between the numerical range and the adjusted evaluation value.

3. The method according to claim 1, wherein: Determining the quality evaluation result of the to-be-evaluated text according to the respective baseline evaluation value and the adjusted evaluation value of the at least one evaluation parameter comprises: For each evaluation parameter of the at least one evaluation parameter, determining a sub-evaluation value of the evaluation parameter according to the baseline evaluation value of the evaluation parameter and the adjusted evaluation value; and The quality evaluation result is determined according to at least one sub-evaluation value of the at least one evaluation parameter.

4. The method according to claim 3, wherein: Determining the quality assessment result according to at least one sub-assessment value of the at least one assessment parameter comprises: determining a total evaluation value based on at least one sub-evaluation value of the at least one evaluation parameter; and The quality grade of the text to be evaluated is determined according to the evaluation value range in which the total evaluation value is located; and the quality evaluation result includes the quality grade.

5. The method according to any one of claims 1 to 4, wherein: The at least one evaluation parameter comprises at least one of the following: The abnormal character ratio represents the ratio of the predetermined abnormal characters to the total number of characters; Word repetition rate, which represents the ratio of words that appear at least twice to the total number of words after deduplication; Word repetition rate, which represents the ratio of words that appear at least twice to the total number of words after deduplication; as well as The language perplexity represents the fluency of the text to be evaluated.

6. The method according to claim 1, wherein: The predetermined conditions include: The total number of characters in the text to be evaluated is not less than a predetermined number threshold; The content category of the text to be evaluated is different from a predetermined abnormal category; The repetition rate of the text to be evaluated is not greater than a predetermined repetition rate threshold; and A ratio between the number of characters associated with a predetermined language and the total number of characters of the text to be evaluated is not less than a first predetermined ratio.

7. The method according to claim 6, wherein: In the case where the text to be evaluated includes a table, the predetermined condition further includes at least one of the following: In the target number of rows of the table, the ratio between the number of columns containing characters of the predetermined language and the total number of columns is not less than a second predetermined ratio; as well as In the target number column of the table, a ratio between the number of rows containing characters of the predetermined language and the total number of rows is not less than a third predetermined ratio.

8. A text quality assessment device, comprising: A first determination module, configured to determine whether the text to be evaluated meets a predetermined condition according to at least one of the total number of characters, content category, repetition rate, and characters related to a predetermined language of the text to be evaluated; A second determination module is used to determine an adjusted evaluation value of at least one evaluation parameter according to a processing logic corresponding to at least one evaluation parameter when it is determined that the text to be evaluated meets the predetermined condition; as well as The third determination module is used to determine the quality evaluation result of the to-be-evaluated text according to the respective baseline evaluation value and the adjusted evaluation value of the at least one evaluation parameter.

9. The device according to claim 8, wherein: The second determining module comprises: A processing submodule, used for processing the text to be evaluated based on the processing logic corresponding to the evaluation parameters to obtain a processing result; The adjusted evaluation value determination submodule is used to determine the adjusted evaluation value of the evaluation parameter according to the numerical range of the processing result and the mapping relationship between the numerical range and the adjusted evaluation value.

10. The device according to claim 8, wherein: The third determination module comprises: a sub-evaluation value determination submodule, configured to determine, for each evaluation parameter of the at least one evaluation parameter, a sub-evaluation value of the evaluation parameter according to the baseline evaluation value of the evaluation parameter and the adjusted evaluation value; and The result determination submodule is used to determine the quality assessment result according to at least one sub-assessment value of the at least one assessment parameter.

11. The device according to claim 10, wherein: The result determination submodule comprises: a total evaluation value determining unit, configured to determine a total evaluation value according to at least one sub-evaluation value of the at least one evaluation parameter; The grade determination unit is used to determine the quality grade of the text to be evaluated according to the evaluation value range in which the total evaluation value is located; the quality evaluation result includes the quality grade.

12. The device according to any one of claims 8 to 11, wherein: The at least one evaluation parameter comprises at least one of the following: The abnormal character ratio represents the ratio of the predetermined abnormal characters to the total number of characters; Word repetition rate, which represents the ratio of words that appear at least twice to the total number of words after deduplication; Word repetition rate, which represents the ratio of words that appear at least twice to the total number of words after deduplication; as well as The language perplexity represents the fluency of the text to be evaluated.

13. The device according to claim 8, wherein: The predetermined conditions include: The total number of characters in the text to be evaluated is not less than a predetermined number threshold; The content category of the text to be evaluated is different from a predetermined abnormal category; The repetition rate of the text to be evaluated is not greater than a predetermined repetition rate threshold; and A ratio between the number of characters associated with a predetermined language and the total number of characters of the text to be evaluated is not less than a first predetermined ratio.

14. The device according to claim 13, wherein: In the case where the text to be evaluated includes a table, the predetermined condition further includes at least one of the following: In the target number of rows of the table, the ratio between the number of columns containing characters of the predetermined language and the total number of columns is not less than a second predetermined ratio; as well as In the target number column of the table, a ratio between the number of rows containing characters of the predetermined language and the total number of rows is not less than a third predetermined ratio.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images