Text corpus scoring method and device, equipment and storage medium

Abstract

The invention discloses a text corpus scoring method and device, equipment and a storage medium. The text corpus scoring method comprises the following steps: in response to an obtained to-be-scored text, performing text segmentation on the to-be-scored text to obtain a minimum scoring unit; aiming at each minimum score unit, filtering by adopting at least two heuristic filters to obtain a comprehensive filtering result of each minimum score unit; the comprehensive filtering result comprises a local filtering result of each heuristic filter for the minimum scoring unit; based on the comprehensive filtering result and the weight of each heuristic filter, determining the unit mass score of the minimum scoring unit; and based on the unit quality score of each minimum scoring unit, determining a comprehensive quality score of the to-be-scored text. According to the technical scheme, the quality of the text corpus can be efficiently determined.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a text corpus scoring method, device, equipment and storage medium. Background Art

[0002] In the process of question-answering model training, the quality of text corpus plays a key role in the final model training effect. If the quality of the collected text corpus is uneven, it will greatly affect the training effect of the model.

[0003] In order to improve the model training effect, the commonly used text corpus scoring method is manual annotation, which is not only inefficient but also consumes a lot of manpower costs. Therefore, how to efficiently determine the corpus quality of each text corpus in a large amount of text corpus is very important to ensure the model training effect. Summary of the invention

[0004] The present invention provides a text corpus scoring method, device, equipment and storage medium to solve the problem of low efficiency in manual screening of text corpus.

[0005] According to one aspect of the present invention, a text corpus scoring method is provided, comprising:

[0006] In response to obtaining the text to be scored, segmenting the text to be scored to obtain a minimum scoring unit;

[0007] For each minimum scoring unit, at least two heuristic filters are used to filter respectively to obtain a comprehensive filtering result of each minimum scoring unit; the comprehensive filtering result includes a local filtering result of each heuristic filter for the minimum scoring unit;

[0008] Determining a unit quality score of a minimum scoring unit based on the comprehensive filtering result and the weight of each heuristic filter;

[0009] Based on the unit quality score of each minimum scoring unit, the comprehensive quality score of the text to be scored is determined.

[0010] According to another aspect of the present invention, a text corpus scoring device is provided, comprising:

[0011] A text segmentation module, configured to segment the text to be scored to obtain a minimum scoring unit in response to obtaining the text to be scored;

[0012] A filtering result determination module, for filtering each minimum scoring unit using at least two heuristic filters to obtain a comprehensive filtering result for each minimum scoring unit; the comprehensive filtering result includes a local filtering result of each heuristic filter for the minimum scoring unit;

[0013] A unit quality score determination module, used to determine the unit quality score of the minimum scoring unit based on the comprehensive filtering result and the weight of each heuristic filter;

[0014] The comprehensive quality score determination module is used to determine the comprehensive quality score of the text to be scored based on the unit quality score of each minimum scoring unit.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the text corpus scoring method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the text corpus scoring method described in any embodiment of the present invention when executed.

[0020] According to another aspect of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the text corpus scoring method of any embodiment of the present disclosure is implemented.

[0021] The technical solution of the embodiment of the present invention, in response to obtaining the text to be scored, performs text segmentation on the text to be scored to obtain minimum scoring units, and then uses at least two heuristic filters to filter each minimum scoring unit to obtain a comprehensive filtering result of each minimum scoring unit, determines the unit quality score of the minimum scoring unit based on the comprehensive filtering result and the weight of each heuristic filter, and finally determines the comprehensive quality score of the text to be scored based on the unit quality score of each minimum scoring unit, scores the minimum scoring unit through the weights and filtering results of multiple heuristic filters, and calculates the quality score of the text to be scored based on the quality score of the minimum scoring unit, thereby solving the problem of low efficiency in manual screening of text corpora and improving the efficiency of quality determination of text corpora.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flowchart of a text corpus scoring method provided according to Embodiment 1 of the present invention;

[0025] Figure 2 is a flowchart of a text corpus scoring method provided according to Embodiment 1 of the present invention;

[0026] Figure 3 is a structural schematic diagram of a text corpus scoring device provided according to Embodiment 3 of the present invention;

[0027] Figure 4 It is a schematic diagram of the structure of an electronic device for implementing the text corpus scoring method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment 1

[0031] Figure 1A flowchart of a text corpus scoring method is provided for the first embodiment of the present invention. This embodiment is applicable to the case where a text to be scored is scored through multiple heuristic filters. The method can be executed by a text corpus scoring device. The text corpus scoring device can be implemented in the form of hardware and / or software. The text corpus scoring device can be configured in various general-purpose computing devices. Figure 1 As shown, the method includes:

[0032] S110 : In response to obtaining the text to be scored, segment the text to be scored to obtain a minimum scoring unit.

[0033] The text to be scored is a text corpus that needs to be scored for quality, wherein the factors that may be considered for quality scoring include text complexity, word repetition ratio, text grammar, text length, and logical coherence. Subsequently, it may be determined whether to use the text to be scored as a training sample for model training based on the final quality score of the text to be scored. Exemplarily, the text to be scored may be a Word document, a PDF file, or a Hypertext Markup Language (HTML) file.

[0034] The minimum scoring unit is obtained by segmenting the text to be scored, and illustratively, the minimum scoring unit may be a page, a paragraph, a sentence, or a phrase, etc. The minimum scoring unit may be flexibly set according to actual scoring calculation requirements, and is not specifically limited in this embodiment.

[0035] In the embodiment of the present invention, since the overall length of the text to be scored is relatively large, if the quality score is performed as a whole, the amount of calculation will be very large. In order to reduce the amount of calculation and improve the efficiency of quality scoring, the text to be scored can be first segmented according to the segmentation rules to obtain the minimum scoring unit.

[0036] Optionally, the text to be scored is segmented to obtain a minimum scoring unit, including at least one of the following:

[0037] According to the sentence end mark in the text to be rated, the text to be rated is segmented into sentences, and the obtained sentences are used as the minimum scoring unit;

[0038] According to the paragraph marks in the text to be rated, the text to be rated is segmented into paragraphs, and the obtained paragraphs are used as the minimum scoring unit;

[0039] According to the phrase end markers in the text to be rated, the text to be rated is segmented into phrases, and the obtained phrases are used as the minimum scoring units.

[0040] In this optional embodiment, the text to be scored is segmented into sentences according to sentence ending marks in the text to be scored, such as periods in word documents or end tags in HTML files, and the obtained sentences are used as the minimum scoring units; the text to be scored can also be segmented into paragraphs according to paragraph marks in the text to be scored, such as first line indents or line breaks, and the obtained paragraphs are used as the minimum scoring units; the text to be scored can also be segmented into phrases according to phrase ending marks in the text to be scored, such as commas or semicolons, and the obtained phrases are used as the minimum scoring units.

[0041] S120 , for each minimum scoring unit, use at least two heuristic filters to perform filtering respectively, to obtain a comprehensive filtering result for each minimum scoring unit; the comprehensive filtering result includes a local filtering result of each heuristic filter for the minimum scoring unit.

[0042] A heuristic filter is a data screening tool based on preset rules or heuristic methods. Exemplarily, the heuristic filter can be at least one of a text complexity filter, a word repetition ratio filter, a text grammar filter, a text length filter, a sentiment analysis filter, a topic consistency filter, a logical coherence filter, a word diversity filter, a cultural sensitivity filter, a text readability filter, an information density filter, an information timeliness filter, an information accuracy filter, and a semantic cleanliness filter.

[0043] In an embodiment of the present invention, after the text to be scored is segmented to obtain the minimum scoring unit, at least two heuristic filters are used for filtering for each minimum scoring unit, and each heuristic filter outputs a local filtering result for the current minimum scoring unit. The matrix composed of the local filtering results of all heuristic filters is used as the comprehensive filtering result of the current minimum scoring unit. Among them, the local filtering result can be a Boolean value, that is, if the current minimum scoring unit can meet the screening conditions of the heuristic filter, the heuristic filter outputs 1, otherwise it outputs 0. The local filtering result can also be a score between 0 and 1. The larger the score, the more the current minimum scoring unit can meet the screening conditions of the heuristic filter.

[0044] In a specific example, for each minimum scoring unit, multiple pre-set text complexity filters, word repetition ratio filters and text grammar filters are used for filtering, and the comprehensive filtering result of the minimum scoring unit is {0,1,0}, that is, the current minimum scoring unit meets the filtering conditions of the word repetition ratio filter, but does not meet the filtering conditions of the text complexity filter and the text grammar filter.

[0045] S130 : Determine the unit quality score of the minimum scoring unit based on the comprehensive filtering result and the weight of each heuristic filter.

[0046] In the embodiment of the present invention, the unit quality score of the minimum scoring unit is determined based on the comprehensive filtering result that integrates the results of multiple heuristic filters and the weight of each heuristic filter. For example, the unit quality score of the minimum scoring unit can be obtained by weighted summing the local filtering results in the comprehensive filtering result based on the weight of the heuristic filter.

[0047] The weight of the heuristic filter can be determined in advance based on the characteristics of the text corpus required for model training. For example, when the model is trained, there is a high requirement for the diversity of words in the text corpus, and the weight of the word diversity filter is increased accordingly. The sum of the weights of all heuristic filters used in the scoring process is 1.

[0048] S140: Determine the comprehensive quality score of the text to be scored based on the unit quality score of each minimum scoring unit.

[0049] In the embodiment of the present invention, the comprehensive quality score of the text to be rated is finally determined based on the unit quality score of each minimum rating unit. Specifically, the unit quality scores of all minimum rating units can be summed to obtain the comprehensive quality score of the text to be rated; the unit quality scores of each minimum rating unit can also be weighted and summed to obtain the comprehensive quality score of the text to be rated, wherein the weight can be determined according to the position of the minimum rating unit in the text to be rated, for example, the more forward the minimum rating unit is, the greater the weight; it can also be determined according to the similarity between the minimum rating unit and the summary of the text to be rated, the higher the similarity, the greater the weight of the minimum rating unit.

[0050] The technical solution of the embodiment of the present invention, in response to obtaining the text to be scored, performs text segmentation on the text to be scored to obtain minimum scoring units, and then uses at least two heuristic filters to filter each minimum scoring unit to obtain a comprehensive filtering result of each minimum scoring unit, determines the unit quality score of the minimum scoring unit based on the comprehensive filtering result and the weight of each heuristic filter, and finally determines the comprehensive quality score of the text to be scored based on the unit quality score of each minimum scoring unit, scores the minimum scoring unit through the weights and filtering results of multiple heuristic filters, and calculates the quality score of the text to be scored based on the quality score of the minimum scoring unit, thereby solving the problem of low efficiency in manual screening of text corpora and improving the efficiency of quality determination of text corpora.

[0051] Embodiment 2

[0052] Figure 2This is a flowchart of a text corpus scoring method provided in the second embodiment of the present invention. This embodiment further refines the above embodiment and provides specific steps before text segmentation of the text to be scored, as well as specific steps for determining the unit quality score of the minimum scoring unit and determining the comprehensive quality score of the text to be scored. Figure 2 As shown, the method includes:

[0053] S210: Use at least two heuristic filters to filter the text corpus in the original text corpus set to obtain a text corpus subset that meets the filtering conditions of each heuristic filter.

[0054] In the embodiment of the present invention, at least two heuristic filters are respectively used to filter the text corpora in the original text corpus set, and the set of text corpora that meets the filtering conditions of the heuristic filters is used as the text corpus subset associated with the heuristic filters.

[0055] In a specific example, a text complexity filter is used to filter the text corpus in the original text corpus set to obtain a text corpus subset that meets the screening conditions of the text complexity filter, that is, a text corpus subset with higher text complexity; and a logical coherence filter is used to filter the text corpus in the original text corpus set to obtain a text corpus subset that meets the screening conditions of the logical coherence filter, that is, a text corpus subset with higher logical coherence.

[0056] S220, calculating the verification perplexity of the text corpus subset associated with each heuristic filter and the original text corpus set.

[0057] Perplexity is a concept in natural language processing and statistical language modeling, which is used to measure the prediction ability of a language model for test data. The verification perplexity in the embodiment of the present invention is related to the probability of the language model predicting the i-th word given the previous i-1 words.

[0058] In the embodiment of the present invention, the verification perplexity of the text corpus subset associated with each heuristic filter and the verification perplexity of the original text corpus set are calculated. The specific calculation method of the verification perplexity is as follows:

[0059]

[0060] Among them, p θ (x i |x ＜i ) is the probability of predicting the ith text corpus given the corpus before the ith text corpus, X is the test set (the expected subset of text or the original text corpus), and t is the number of text corpora in the test set.

[0061] S230, determining the weight of each heuristic filter based on the verification perplexity of the text corpus subset associated with each heuristic filter and the verification perplexity of the original text corpus set; the weight of the heuristic filter is negatively correlated with the verification perplexity of the associated text corpus subset.

[0062] In an embodiment of the present invention, the weight of each heuristic filter is determined based on the verification perplexity of the text corpus subset associated with each heuristic filter and the verification perplexity of the original text corpus set. The weight of the heuristic filter is negatively correlated with the verification perplexity of the associated text corpus subset. Specifically, setting the weight of the heuristic filter to be negatively correlated with the verification perplexity of the associated text corpus subset can make the heuristic filter with a lower perplexity of the text corpus subset screened out have a higher weight, which is beneficial to improving the reliability of the score. Exemplarily, the verification perplexity is normalized and used as the weight.

[0063] Optionally, the weight of each heuristic filter is determined based on the validation perplexity of the text corpus subset associated with each heuristic filter and the validation perplexity of the original text corpus set, including:

[0064] Calculate the difference between the validation perplexity of the original text corpus and the validation perplexity of the text corpus subset associated with each heuristic filter;

[0065] The ratio between the difference and the validation perplexity of the original text corpus is calculated, and if the ratio is greater than 0, the difference is used as the weight of the heuristic filter.

[0066] The embodiment of the present invention provides a specific method for determining the weight of each heuristic filter based on the verification perplexity of the text corpus subset associated with each heuristic filter and the verification perplexity of the original text corpus set: first, the difference between the verification perplexity of the original text corpus set and the verification perplexity of the text corpus subset associated with each heuristic filter is calculated. Further, the ratio between the difference and the verification perplexity of the original text corpus set is calculated, and when the ratio is greater than 0, the difference is used as the weight of the heuristic filter. Specifically, the weight calculation formula of the heuristic filter is as follows:

[0067]

[0068] Among them, ω i is the weight of the i-th heuristic filter, PPL i is the verification perplexity of the ith heuristic filter associated with the text corpus subset, PPL all is the verification perplexity of the original text corpus set, and the weight lower limit is set to 0 to avoid negative weights.

[0069] S240: In response to obtaining the text to be scored, segment the text to be scored to obtain a minimum scoring unit.

[0070] S250: For each minimum scoring unit, use at least two heuristic filters to perform filtering to obtain a comprehensive filtering result for each minimum scoring unit; the comprehensive filtering result includes a local filtering result of each heuristic filter for the minimum scoring unit.

[0071] S260: Determine the unit quality score of the minimum scoring unit based on the comprehensive filtering result and the weight of each heuristic filter.

[0072] Optionally, the unit quality score of the minimum scoring unit is calculated as follows:

[0073]

[0074] Among them, Score line is the unit quality score of the smallest scoring unit, ω i is the weight of the i-th heuristic filter, F is the number of heuristic filters used, and I i (line) is the local filtering result of the i-th heuristic filter for the current minimum scoring unit.

[0075] In this optional embodiment, a method for calculating the unit quality score of the minimum scoring unit is provided. The unit quality score of the current minimum scoring unit is calculated based on the weight of the heuristic filter and the local filtering results of each heuristic filter for the minimum scoring unit. This can make the more important heuristic filters have higher weights, thereby improving the reliability of the scoring.

[0076] S270: Determine the comprehensive quality score of the text to be scored based on the unit quality score of each minimum scoring unit.

[0077] Optionally, the comprehensive quality score of the text to be rated is calculated as follows:

[0078]

[0079] Among them, Score doc is the comprehensive quality score of the text to be rated, t line is the weight of the smallest scoring unit in the text to be scored, Score line is the unit quality score of the minimum scoring unit, and n is the number of minimum scoring units contained in the text to be scored.

[0080] In this optional embodiment, a method for calculating the comprehensive quality score of the text to be rated is provided, and the unit quality scores of the minimum rating units contained in the text to be rated are weighted and summed, so that the comprehensive quality score of the text to be rated can be efficiently determined. Among them, the weight of each minimum rating unit can be determined according to the position of the minimum rating unit in the text to be rated, or it can be determined according to the similarity between the minimum rating unit and the summary of the text to be rated. The weight distribution method can be flexibly adjusted according to actual needs to improve the flexibility of the rating calculation.

[0081] Optionally, the weight of the minimum scoring unit in the text to be scored is calculated as follows:

[0082]

[0083] Among them, z i is the similarity between the i-th minimum scoring unit and the summary of the text to be scored, and n is the number of minimum scoring units contained in the text to be scored.

[0084] In this optional embodiment, a method for calculating the weight of the minimum scoring unit in the text to be scored is provided. First, the summary of the text to be scored is obtained, and then the similarity between the minimum scoring unit and the summary is calculated. Finally, a softmax normalization operation is performed to obtain the weight of the minimum scoring unit in the text to be scored, so as to increase the weight of important text in the text to be scored, thereby improving the reliability of the scoring.

[0085] The technical solution of the embodiment of the present invention determines the weight of the heuristic filter according to the verification perplexity of the text corpus subset associated with each heuristic filter, and determines the unit quality score of the minimum scoring unit according to the comprehensive filtering result of the minimum scoring unit and the weight of each heuristic filter. Finally, based on the unit quality score of each minimum scoring unit, the comprehensive quality score of the text to be scored is determined, which can improve the scoring reliability.

[0086] Embodiment 3

[0087] Figure 3 A schematic diagram of the structure of a text corpus scoring device provided in Embodiment 3 of the present invention.

[0088] like Figure 3 As shown, the device comprises:

[0089] A text segmentation module 310 is used to segment the text to be scored in response to obtaining the text to be scored to obtain a minimum scoring unit;

[0090] The filtering result determination module 320 is used to filter each minimum scoring unit using at least two heuristic filters to obtain a comprehensive filtering result of each minimum scoring unit; the comprehensive filtering result includes a local filtering result of each heuristic filter for the minimum scoring unit;

[0091] A unit quality score determination module 330, for determining a unit quality score of a minimum scoring unit based on the comprehensive filtering result and the weight of each heuristic filter;

[0092] The comprehensive quality score determination module 340 is used to determine the comprehensive quality score of the to-be-scored text based on the unit quality score of each minimum scoring unit.

[0093] The technical solution of the embodiment of the present invention, in response to obtaining the text to be scored, performs text segmentation on the text to be scored to obtain minimum scoring units, and then uses at least two heuristic filters to filter each minimum scoring unit to obtain a comprehensive filtering result of each minimum scoring unit, determines the unit quality score of the minimum scoring unit based on the comprehensive filtering result and the weight of each heuristic filter, and finally determines the comprehensive quality score of the text to be scored based on the unit quality score of each minimum scoring unit, scores the minimum scoring unit through the weights and filtering results of multiple heuristic filters, and calculates the quality score of the text to be scored based on the quality score of the minimum scoring unit, thereby solving the problem of low efficiency in manual screening of text corpora and improving the efficiency of quality determination of text corpora.

[0094] Optionally, the text corpus scoring device further includes:

[0095] A text corpus screening module is used to screen the text corpus in the original text corpus set using at least two heuristic filters before segmenting the text to be scored, so as to obtain a text corpus subset that meets the screening conditions of each heuristic filter;

[0096] A perplexity calculation module, used to calculate the verification perplexity of the text corpus subset associated with each heuristic filter and the original text corpus set;

[0097] The filter weight determination module is used to determine the weight of each heuristic filter based on the verification perplexity of the text corpus subset associated with each heuristic filter and the verification perplexity of the original text corpus set; the weight of the heuristic filter is negatively correlated with the verification perplexity of the associated text corpus subset.

[0098] Optionally, a filter weight determination module is specifically used to:

[0099] Calculate the difference between the validation perplexity of the original text corpus and the validation perplexity of the text corpus subset associated with each heuristic filter;

[0100] The ratio between the difference and the verification perplexity of the original text corpus set is calculated, and when the ratio is greater than 0, the difference is used as the weight of the heuristic filter.

[0101] Optionally, the unit quality score of the minimum scoring unit is calculated as follows:

[0102]

[0103] Among them, Score line is the unit quality score of the smallest scoring unit, ω i is the weight of the i-th heuristic filter, F is the number of heuristic filters used, and I i (line) is the local filtering result of the i-th heuristic filter for the current minimum scoring unit.

[0104] Optionally, the comprehensive quality score of the text to be rated is calculated as follows:

[0105]

[0106] Among them, Score doc is the comprehensive quality score of the text to be rated, t line is the weight of the smallest scoring unit in the text to be scored, Score line is the unit quality score of the minimum scoring unit, and n is the number of minimum scoring units contained in the text to be scored.

[0107] Optionally, the weight of the minimum scoring unit in the text to be scored is calculated as follows:

[0108]

[0109] Among them, z i is the similarity between the i-th minimum scoring unit and the summary of the text to be scored, and n is the number of minimum scoring units contained in the text to be scored.

[0110] Optionally, the heuristic filter includes at least one of a text complexity filter, a word repetition ratio filter, a text grammar filter, a text length filter, a sentiment analysis filter, a topic consistency filter, a logical coherence filter, a word diversity filter, a cultural sensitivity filter, a text readability filter, an information density filter, an information timeliness filter, an information accuracy filter, and a semantic cleanliness filter.

[0111] The text corpus scoring device provided in the embodiment of the present invention can execute the text corpus scoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0112] In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0113] Embodiment 4

[0114] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0115] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, applicators, blade applicators, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0116] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0117] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0118] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a text corpus scoring method.

[0119] In some embodiments, the text corpus scoring method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the text corpus scoring method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the text corpus scoring method in any other appropriate manner (e.g., by means of firmware).

[0120] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or application.

[0122] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0123] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0124] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data applier), or a computing system that includes a middleware component (e.g., an application applier), or a computing system that includes a frontend component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0125] A computing system may include a client and an application server. The client and the application server are generally remote from each other and usually interact through a communication network. The relationship between the client and the application server is generated by computer programs running on corresponding computers and having a client-application server relationship with each other. The application server may be a cloud application server, also known as a cloud computing application server or a cloud host, which is a host product in a cloud computing application system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS applications.

[0126] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0127] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A text corpus scoring method, characterized in that: include: In response to obtaining the text to be scored, segmenting the text to be scored to obtain a minimum scoring unit; For each minimum scoring unit, at least two heuristic filters are used to filter respectively to obtain a comprehensive filtering result of each minimum scoring unit; the comprehensive filtering result includes a local filtering result of each heuristic filter for the minimum scoring unit; Determining a unit quality score of a minimum scoring unit based on the comprehensive filtering result and the weight of each heuristic filter; Based on the unit quality score of each minimum scoring unit, the comprehensive quality score of the text to be scored is determined.

2. The method according to claim 1, characterized in that: Before segmenting the text to be scored, the method further includes: Using at least two heuristic filters to filter the text corpus in the original text corpus set, respectively, to obtain a text corpus subset that meets the screening conditions of each heuristic filter; Compute the validation perplexity for the subset of text corpus associated with each heuristic filter and the original text corpus set; The weight of each heuristic filter is determined based on the verification perplexity of the text corpus subset associated with each heuristic filter and the verification perplexity of the original text corpus set; the weight of the heuristic filter is negatively correlated with the verification perplexity of the associated text corpus subset.

3. The method according to claim 2, characterized in that The weight of each heuristic filter is determined based on the validation perplexity of the text corpus subset associated with each heuristic filter and the validation perplexity of the original text corpus set, including: Calculate the difference between the validation perplexity of the original text corpus and the validation perplexity of the text corpus subset associated with each heuristic filter; The ratio between the difference and the verification perplexity of the original text corpus set is calculated, and when the ratio is greater than 0, the difference is used as the weight of the heuristic filter.

4. The method according to claim 1, characterized in that: The unit quality score of the smallest scoring unit is calculated as follows: Among them, Score line is the unit quality score of the smallest scoring unit, ω i is the weight of the i-th heuristic filter, F is the number of heuristic filters used, and I i (line) is the local filtering result of the i-th heuristic filter for the current minimum scoring unit.

5. The method according to claim 1, characterized in that The comprehensive quality score of the text to be rated is calculated as follows: Among them, Score doc is the comprehensive quality score of the text to be rated, t line is the weight of the smallest scoring unit in the text to be scored, Score line is the unit quality score of the minimum scoring unit, and n is the number of minimum scoring units contained in the text to be scored.

6. The method according to claim 5, characterized in that The weight of the smallest scoring unit in the text to be scored is calculated as follows: Among them, z i is the similarity between the i-th minimum scoring unit and the summary of the text to be scored, and n is the number of minimum scoring units contained in the text to be scored.

7. The method according to any one of claims 1 to 6, characterized in that: The heuristic filter includes at least one of a text complexity filter, a word repetition ratio filter, a text grammar filter, a text length filter, a sentiment analysis filter, a topic consistency filter, a logical coherence filter, a word diversity filter, a cultural sensitivity filter, a text readability filter, an information density filter, an information timeliness filter, an information accuracy filter, and a semantic cleanliness filter.

8. A text corpus scoring device, characterized in that: include: A text segmentation module, configured to segment the text to be scored to obtain a minimum scoring unit in response to obtaining the text to be scored; A filtering result determination module, for filtering each minimum scoring unit using at least two heuristic filters to obtain a comprehensive filtering result for each minimum scoring unit; the comprehensive filtering result includes a local filtering result of each heuristic filter for the minimum scoring unit; A unit quality score determination module, used to determine the unit quality score of the minimum scoring unit based on the comprehensive filtering result and the weight of each heuristic filter; The comprehensive quality score determination module is used to determine the comprehensive quality score of the text to be scored based on the unit quality score of each minimum scoring unit.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the text corpus scoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the text corpus scoring method according to any one of claims 1 to 7 when executed.

Images