Expanded text score determination method and device, equipment and storage medium
By acquiring and analyzing the score information of the sentence to be expanded, and combining it with the score level information, the score of the expanded text is automatically determined, which solves the problem that the score of the expanded text cannot be automatically determined, and achieves efficient and accurate text scoring.
Patent Information
- Application Number
- CN202310598473.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-24
- Publication Date
- 2024-11-26
AI Technical Summary
Existing pronunciation assessment technologies cannot automatically determine the score of the expanded text after data augmentation, and manual annotation is labor-intensive and inaccurate.
By acquiring the first score information of the sentence to be expanded, and combining it with the score level information, the comprehensive accuracy score of the first sentence and the comprehensive accuracy score of the first word are automatically determined. The sentence to be expanded is then expanded to generate the target sentence text. For each target sentence text, a second word accuracy score is determined, thus achieving automatic determination of the score of the expanded text.
It eliminates the need for manual scoring, saving manpower, improving efficiency, and increasing accuracy. It can quickly generate a large number of target sentence texts and their scores, thus improving scoring speed.
Smart Images

Figure CN119025664A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment and storage medium for determining a score of an expanded text. Background Art
[0002] Pronunciation assessment technology is a sub-direction of computer-assisted language learning. Pronunciation assessment technology requires efficient and accurate indication of learners' pronunciation accuracy, fluency, completeness and other indicators. Existing pronunciation assessment technologies all require pronunciation error data to fit the scoring model, but the annotation of pronunciation error data is very difficult and large-scale production is expensive. For example: the existing open source dataset speechocean762 for learning English has only 5,000 sentences (including recordings of 250 people, an average of 20 per person), and each sentence is annotated with sentence accuracy, fluency, and completeness; each word is annotated with accuracy and whether the stress is correct; each phoneme in the word is also annotated with accuracy; it can be seen that the annotation is very complicated and requires professional English knowledge. In addition, there is a certain degree of subjectivity in the scoring of the same audio by multiple English experts, and sometimes there are even large differences, which leads to insufficient accuracy of the scores.
[0003] In the case of a small amount of erroneous data, it is extremely necessary to automatically expand the data. In the prior art, after expanding the text data, the pronunciation score of the expanded data is not explained, and the score of the expanded text cannot be automatically determined; if manual annotation is used, the workload is large, and due to subjectivity, there may be inaccurate scores. Summary of the invention
[0004] The present invention provides a method, device, equipment and storage medium for determining the score of an expanded text, so as to solve the problem that the score cannot be automatically determined after data expansion, and the manual determination of the score is labor-intensive and inaccurate.
[0005] According to one aspect of the present invention, a method for determining a score of an expanded text is provided, comprising:
[0006] Acquire a sentence to be expanded and first score information of the sentence to be expanded, wherein the first score information includes at least one score set, and the score set includes a first sentence accuracy score and at least one first word accuracy score;
[0007] Determine a first sentence accuracy comprehensive score and a first word accuracy comprehensive score according to the first score information and the score gear information of the sentence to be expanded;
[0008] Expanding the sentence to be expanded to generate at least one target sentence text;
[0009] For each target sentence text, determine at least one second word accuracy score corresponding to the target sentence text, and determine the second sentence accuracy composite score of the target sentence text based on each of the second word accuracy scores, each of the first word accuracy composite scores, and the first sentence accuracy composite score.
[0010] According to another aspect of the present invention, there is provided a device for determining a score of an expanded text, comprising:
[0011] A first score information acquisition module, used to acquire a sentence to be expanded and first score information of the sentence to be expanded, wherein the first score information includes at least one score set, and the score set includes a first sentence accuracy score and at least one first word accuracy score;
[0012] An accuracy comprehensive score module, used to determine a first sentence accuracy comprehensive score and a first word accuracy comprehensive score according to the first score information and score gear information of the sentence to be expanded;
[0013] A target sentence generating module, used for expanding the sentence to be expanded to generate at least one target sentence text;
[0014] The target sentence score determination module is used to determine, for each target sentence text, at least one second word accuracy score corresponding to the target sentence text, and determine the second sentence accuracy comprehensive score of the target sentence text based on each of the second word accuracy scores, each of the first word accuracy comprehensive scores and the first sentence accuracy comprehensive score.
[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0016] at least one processor; and
[0017] a memory communicatively connected to the at least one processor; wherein,
[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method for determining a score of an expanded text according to any embodiment of the present invention.
[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining a score of an expanded text according to any embodiment of the present invention when executed.
[0020] The technical solution of the embodiment of the present invention comprises: obtaining a sentence to be expanded and first score information of the sentence to be expanded, wherein the first score information comprises at least one score set, wherein the score set comprises a first sentence accuracy score and at least one first word accuracy score; determining a first sentence accuracy comprehensive score and a first word accuracy comprehensive score according to the first score information and score gear information of the sentence to be expanded; expanding the sentence to be expanded to generate at least one target sentence text; determining at least one second word accuracy score corresponding to the target sentence text for each target sentence text, and determining the second sentence accuracy score of the target sentence text according to each of the second word accuracy scores, each of the first word accuracy comprehensive scores and the first sentence accuracy comprehensive score. The comprehensive score solves the problem that the score of the expanded text cannot be automatically determined. The first score information of the sentence to be expanded is analyzed and processed in combination with the score gear information to determine the first sentence accuracy comprehensive score and the first word accuracy comprehensive score, thereby ensuring the accuracy of the score of the sentence to be expanded; the sentence to be expanded is expanded to generate a target sentence text, and the second word accuracy score of each word in the target sentence text is determined. The second sentence accuracy comprehensive score of the target sentence text is determined by the second word accuracy score, the first word accuracy comprehensive score and the first sentence accuracy comprehensive score, thereby realizing automatic determination of the score of the expanded text without manual scoring, saving human resources and improving efficiency. In addition, the application determines the comprehensive score of the accuracy of sentences and words, which is more accurate.
[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0023] Figure 1 is a flowchart of a method for determining a score of an expanded text provided in Embodiment 1 of the present invention;
[0024] Figure 2 is a flowchart of a method for determining a score of an expanded text provided in Embodiment 2 of the present invention;
[0025] Figure 3is a schematic diagram of the structure of a device for determining a score of an expanded text provided in Embodiment 3 of the present invention;
[0026] Figure 4 It is a structural schematic diagram of an electronic device for implementing the method for determining a score of an expanded text according to an embodiment of the present invention. DETAILED DESCRIPTION
[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0029] Embodiment 1
[0030] Figure 1 A flowchart of a method for determining the score of an expanded text is provided for the first embodiment of the present invention. This embodiment is applicable to the case of determining the evaluation score of an expanded text. The method can be performed by a device for determining the score of an expanded text. The device for determining the score of an expanded text can be implemented in the form of hardware and / or software. The device for determining the score of an expanded text can be configured in an electronic device. Figure 1 As shown, the method includes:
[0031] S101. Obtain a sentence to be expanded and first score information of the sentence to be expanded, where the first score information includes at least one score set, and the score set includes a first sentence accuracy score and at least one first word accuracy score.
[0032] In this embodiment, the sentence to be expanded can be specifically understood as a sentence with expansion requirements, and the sentence to be expanded can include both text and voice forms; the first score information can be specifically understood as the scoring information of the sentence to be expanded, and the score can be a score from multiple different angles, for example, word accuracy, sentence accuracy, sentence completeness, etc. The first score information may include scores of multiple experts from different angles on the expanded sentence; the first score information in this embodiment includes at least one score set, each score set includes a score of an expert's pronunciation evaluation of the expanded sentence, that is, a first sentence accuracy score, and a first word accuracy score corresponding to each word. Among them, the first sentence accuracy score is used to describe the pronunciation accuracy of the entire sentence to be expanded, and the first word accuracy score is used to describe the pronunciation accuracy of each word in the sentence to be expanded.
[0033] The embodiment of the present application can expand any sentence to be expanded and determine the score of the expanded sentence. The number of sentences to be expanded can be one or more. Generate the sentence to be expanded in advance, or select any sentence from the existing data set as the sentence to be expanded. When the sentence to be expanded is obtained, obtain the first score information of the sentence to be expanded. The first score information includes the score of at least one expert from multiple angles. In order to ensure the accuracy of the score and avoid excessive workload, the number of experts, that is, the number of score sets, needs to be reasonably set. For example, the number of score sets can be set to 5, which can both ensure the accuracy of the score and avoid excessive workload.
[0034] S102: Determine a first sentence accuracy comprehensive score and a first word accuracy comprehensive score according to the first score information and the score gear information of the sentence to be expanded.
[0035] In this embodiment, the score gear information can be specifically understood as the information of the gears of different scores; the first sentence accuracy comprehensive score can be specifically understood as the comprehensive score of the sentence accuracy of the sentence to be expanded; the first word accuracy comprehensive score can be specifically understood as the comprehensive score of the accuracy of the words in the sentence to be expanded.
[0036] The score gear information is determined in advance. For example, the full score is 10 points, which is divided into 5 gears from 1 to 5, each gear corresponding to a score range. The gear of the score is determined based on each score contained in the first score information of the sentence to be expanded and combined with the score gear information, and a comprehensive score is calculated based on the gear. When calculating the comprehensive score, the first sentence accuracy comprehensive score is calculated based on the first sentence accuracy score, and the first word accuracy comprehensive score is calculated based on the first word accuracy score. For example, the score calculation method can be determined according to the gear, and then the comprehensive score is calculated according to the score calculation method.
[0037] S103: Expand the sentence to be expanded to generate at least one target sentence text.
[0038] In this embodiment, the target sentence text may be specifically understood as a new sentence text generated after the sentence to be expanded is expanded.
[0039] Specifically, the text corresponding to the sentence to be expanded is expanded, for example, one or more words in the sentence to be expanded are replaced. When replacing multiple words, the number of replaced words can be preset or randomly generated. The maximum number of replaced words can be set to avoid replacing too many words, resulting in a completely different entire sentence. The embodiment of the present application replaces words to generate at least one target sentence text. When generating multiple target sentence texts, different numbers of words can be replaced in sequence to generate corresponding target sentence texts. When replacing the same number of words, the same word can be replaced, and different new words can be used to replace the same word to generate the target sentence text. Words in different positions can also be replaced to ensure that the generated target sentence text is different from the sentence to be expanded.
[0040] S104. For each target sentence text, determine at least one second word accuracy score corresponding to the target sentence text, and determine the second sentence accuracy comprehensive score of the target sentence text according to each second word accuracy score, each first word accuracy comprehensive score and the first sentence accuracy comprehensive score.
[0041] For each target sentence text, its score is determined in the same way. The second word accuracy score corresponding to each word in the target sentence text is determined according to the first word accuracy score corresponding to each word in the sentence to be expanded. For example, the first word accuracy score can be directly used as the second word accuracy score, or the first word accuracy score can be calculated to obtain the second word accuracy score, and so on. The weight or ratio is determined according to each second word accuracy score and the first word accuracy score, and the weight or ratio is multiplied by the first sentence accuracy comprehensive score to obtain the second sentence accuracy comprehensive score of the target sentence text. The way to determine the weight or ratio can be to calculate the total score of the first word accuracy comprehensive score and the total score of the second word accuracy score, and use the ratio of the two total scores as the weight or ratio; or, respectively calculate the ratio of the first word accuracy comprehensive score and the second word accuracy score corresponding to each word, perform average, sum, weighted and other operations on each ratio, and use the value obtained as the weight or ratio.
[0042] The method provided in the embodiment of the present application can automatically determine the score of the expanded text, without manual annotation, saving manpower and high efficiency. After obtaining the score corresponding to the target sentence text, it can be saved accordingly, and the voice of the sentence to be expanded is stored as the voice of the target sentence text. The embodiment of the present application can store the voice, comprehensive score of the sentence to be expanded obtained, and the voice and comprehensive score of the target sentence text in a data set for training a pronunciation evaluation model.
[0043] The embodiment of the present invention provides a method for determining the score of an expanded text, which solves the problem that the score of the expanded text cannot be automatically determined. The first score information of the sentence to be expanded is analyzed and processed in combination with the score gear information, and the first sentence accuracy comprehensive score and the first word accuracy comprehensive score are determined to ensure the accuracy of the score of the sentence to be expanded; the sentence to be expanded is expanded to generate a target sentence text, and the second word accuracy score of each word in the target sentence text is determined. The second sentence accuracy comprehensive score of the target sentence text is determined by the second word accuracy score, the first word accuracy comprehensive score and the first sentence accuracy comprehensive score, so that the score of the expanded text is automatically determined without manual scoring, saving human resources and improving efficiency. In addition, the present application determines the comprehensive score of the accuracy of sentences and words, which is more accurate.
[0044] Embodiment 2
[0045] Figure 2 This is a flowchart of a method for determining the score of an expanded text provided in the second embodiment of the present invention. This embodiment is refined on the basis of the above embodiment. Figure 2 As shown, the method includes:
[0046] S201. Obtain a sentence to be expanded and first score information of the sentence to be expanded, where the first score information includes at least one score set, and the score set includes a first sentence accuracy score and at least one first word accuracy score.
[0047] For example, the text corresponding to the sentence to be expanded is: How are you
[0048] Audio corresponding to the sentence to be expanded: audio of the above text being read aloud, which is not shown here.
[0049] The first score information corresponding to the sentence to be expanded is as follows (taking 5 expert scoring and 5 score sets as an example):
[0050] Expert 1:
[10] , [10, 10, 10]
[0051] Expert 2: [9], [9, 10, 9] ...
[0053] Expert 5: [6], [6, 7, 6]
[0054] The first [ ] is the accuracy score of the first sentence, and the second [ ] is the accuracy score of the first word corresponding to each word. The full score here is 10 points.
[0055] S202: Determine the accuracy span information of the sentence to be expanded according to the accuracy scores of the first sentences corresponding to the sentence to be expanded and the score span information.
[0056] In this embodiment, the accuracy cross-level information can be specifically understood as the level difference information of different levels of accuracy of the sentence to be expanded when it is scored by different experts.
[0057] Specifically, determine the accuracy scores of all first sentences corresponding to the sentence to be expanded, each first sentence accuracy score is scored by an expert, determine the score level information in advance, determine the level where the first sentence accuracy score is located according to the score level information, perform statistical analysis on each level, and determine the accuracy level information.
[0058] As an optional embodiment of this embodiment, this optional embodiment further determines the accuracy span information of the sentence to be expanded according to the accuracy scores of each first sentence corresponding to the sentence to be expanded and the score span information, and optimizes it as follows:
[0059] A1. Determine the accuracy level corresponding to each first sentence accuracy score according to the score level information.
[0060] In this embodiment, the accuracy level can be specifically understood as the level to which the accuracy score belongs. For each first sentence accuracy score, the accuracy score is compared with the score level information to determine the level that matches the first sentence accuracy score and use it as the accuracy level.
[0061] Illustratively, Table 1 provides a reference criterion for sentence scoring, which includes score level information, where the levels include 1-5 and the scores include 0-10 points.
[0062] Table 1 Reference criteria for sentence scoring A2. Determine whether the accuracy gears are the same gear. If so, determine the accuracy span information as the first preset value; otherwise, determine the lowest gear and the highest gear, and add the gear difference between the lowest gear and the highest gear to the first preset value to determine the accuracy span information.
[0063] In this embodiment, the first preset value can be set as required, for example, 0, 1, etc. The first preset value is set in advance to determine whether each accuracy gear is the same gear. If so, the accuracy span information is determined to be the first preset value. If it is not the same gear, all accuracy gears are compared, the lowest gear and the highest gear among all accuracy gears are determined, and the gear difference is calculated. For example, the gears decrease from 1 to 5, the lowest gear is 3, the highest gear is 1, the gear difference is 3-1=2, the first preset value is 1, and the accuracy span information is 3.
[0064] As an optional embodiment of the present embodiment, the further optimization of the present optional embodiment includes that after determining the accuracy level corresponding to the accuracy score of each first sentence according to the score level information, if the accuracy level corresponding to the sentence to be expanded meets the first filtering condition, the sentence to be expanded is eliminated.
[0065] In this embodiment, the first filtering condition can be specifically understood as a condition for filtering the sentences to be expanded by the accuracy gear, which can be preset according to the needs. Exemplarily, when the accuracy gears are distributed in different gears, it is determined that the first filtering condition is met (for example, there are 4 accuracy gears in total, namely gear 1, gear 2, gear 3 and gear 4, and each accuracy gear is distributed in different gears); or, the first filtering condition is met when each accuracy gear covers all gears (for example, there are 6 accuracy gears in total, namely gear 1, gear 2, gear 3, gear 4, gear 3, gear 5, and each accuracy gear covers all gears 1-5). After determining the accuracy gears corresponding to the sentences to be expanded, it is determined whether each accuracy gear meets the first filtering condition. If so, the sample is considered invalid and the sentences to be expanded are eliminated.
[0066] S203: Determine a comprehensive score of the first sentence accuracy based on the accuracy cross-file information and the accuracy scores of each first sentence.
[0067] The comprehensive score calculation method corresponding to different accuracy span information is determined in advance. After the accuracy span information is determined, the calculation method is determined according to the accuracy span information, and the accuracy scores of each first sentence are calculated according to the corresponding calculation method to obtain the first sentence accuracy comprehensive score. The calculation method can be calculating the average value, the maximum value, the minimum value, the weighted sum, etc.
[0068] As an optional embodiment of this embodiment, this optional embodiment further determines the first sentence accuracy comprehensive score according to the accuracy cross-file information combined with the accuracy scores of each first sentence, and optimizes it to:
[0069] B1. When the accuracy span information is within a first preset range, the average of the accuracy scores of the first sentences is determined as the first sentence accuracy comprehensive score.
[0070] In this embodiment, the first preset range can be set according to the requirements, for example, P≤2, where P is the accuracy span information. When the accuracy span information is within the first preset range, the average of the accuracy scores of the first sentences is calculated, and the average is determined as the first sentence accuracy comprehensive score.
[0071] B2. When the accuracy span information is within the second preset range, the accuracy scores of the first sentences are filtered to determine at least one candidate sentence accuracy score, and the average of the accuracy scores of the candidate sentences is determined as the first sentence comprehensive accuracy score.
[0072] In this embodiment, the second preset range can be 3≤P≤5, where P is the accuracy span information. The accuracy scores of the candidate sentences can be specifically understood as alternative scores for calculating the comprehensive score. When the accuracy span information is within the second preset range, the accuracy scores of the first sentences are screened and filtered according to the preset rules to obtain at least one accuracy score of the candidate sentences. The preset rules can be to filter out the highest score and the lowest score. Filtering the highest score and the lowest score can be to filter out one highest score and one lowest score, or to filter out all the highest scores and all the lowest scores; filter out the highest and lowest scores, and so on. Calculate the average of the accuracy scores of the candidate sentences, and determine this average as the comprehensive accuracy score of the first sentence.
[0073] S204. For each word in the sentence to be expanded, determine a comprehensive first word accuracy score according to the accuracy cross-file information and the first word accuracy scores corresponding to the word.
[0074] For each word in the sentence to be expanded, the first word accuracy comprehensive score corresponding to the word is calculated, the first word accuracy scores corresponding to the word are determined, the calculation method is determined according to the accuracy cross-file information, and the first word accuracy scores are calculated according to the corresponding calculation method to obtain the first word accuracy comprehensive score. The calculation method can be to calculate the average value, maximum value, minimum value, weighted sum, etc.
[0075] As an optional embodiment of this embodiment, this optional embodiment further determines the first word accuracy comprehensive score according to the accuracy cross-file information combined with the first word accuracy scores corresponding to the words, and optimizes it to:
[0076] C1. When the accuracy span information is within the first preset range, the average of the first word accuracy scores corresponding to the words is determined as the first word accuracy comprehensive score.
[0077] When the accuracy span information is within the first preset range, the average of the accuracy scores of the first words is calculated, and the average is determined as the comprehensive accuracy score of the first words.
[0078] C2. When the accuracy span information is within the second preset range, the first word accuracy scores corresponding to the words are filtered to determine at least one candidate word accuracy score, and the average of the candidate word accuracy scores is determined as the first word comprehensive accuracy score.
[0079] In this embodiment, the accuracy score of the candidate word can be specifically understood as the candidate score used to calculate the comprehensive score of the word. When the accuracy span information is within the second preset range, the accuracy scores of each first word are screened and filtered according to the preset rules to obtain at least one accuracy score of the candidate word. The preset rules can be to filter out the highest score and the lowest score. Filtering the highest score and the lowest score can be to filter out one highest score and one lowest score, or to filter out all the highest scores and all the lowest scores; filter out the highest and lowest scores, etc. The preset rules can be the same as the rules for filtering the accuracy score of the first sentence, or they can be different. Calculate the average value of the accuracy scores of each candidate word, and determine this average value as the comprehensive accuracy score of the first word.
[0080] The embodiment of the present application can also determine the accuracy span information for sentences and words respectively, that is, the accuracy span information of the sentence to be expanded is determined according to the accuracy scores of each first sentence corresponding to the sentence to be expanded and the score gear information, which is recorded as the first accuracy span information; the accuracy span information of the sentence to be expanded is determined according to the accuracy scores of each first word corresponding to the sentence to be expanded and the score gear information, which is recorded as the second accuracy span information. When determining the second accuracy span information, the corresponding second accuracy span information can be determined for each word, and then the comprehensive score is calculated. The implementation principle is the same as the above-mentioned implementation principle, and the embodiment of the present application will not be repeated.
[0081] Exemplarily, Table 2 provides a reference criterion for word scoring, which includes score level information, where the levels include 1-5 levels and the scores include 0-10 points.
[0082] Table 2 Reference criteria for word scoring
[0083]
[0084] S205: Expand the sentence to be expanded to generate at least one target sentence text.
[0085] The embodiment of the present application can use the language model to replace the original text, generate the target sentence text, and expand the sentence to be expanded. The addition and deletion of words in a sentence will affect the fluency score of the sentence, and the replacement of words in a sentence will affect the accuracy. Taking the expansion of the sentence accuracy score as an example, the words in the sentence are replaced.
[0086] Example: Original text (i.e. the sentence to be expanded): how are you
[0087] Algorithm steps:
[0088] 1. Remove N words from the original text, for example, N = 2, that is, remove are and you from how are you, and replace the removed text with [mask], and the sentence becomes how[mask][mask].
[0089] 2. Use the transform-based model bart to generate the first candidate word [mask]. Set the generation window of beam_search to 2 and exclude the words in the original sentence. Then, how about [mask] and how dare [mask] can be generated.
[0090] 3. Using the transform-based model bart, the input sentences are how about [mask] and how dare [mask], and the candidate words for the second [mask] are generated respectively. The generation window of beam_search is still 2 and the words in the original sentence are still excluded. Then for the first sentence, there is how about this, and for the second sentence, there is how dare he.
[0091] 4. Collect all generated sentences, use a statistical language model to calculate the perplexity score, set a threshold, and only keep the newly generated sentences with perplexity below the threshold. The lower the perplexity, the more it conforms to daily expression.
[0092] After the above algorithm is processed, two target sentence texts can be generated: how about this and how dare he. Similarly, when N=1, two target sentence texts can be generated: how about you and how dare you.
[0093] S206 . For each target sentence text, compare the sentence to be expanded with the target sentence text to determine the difference words and non-difference words in the target sentence text.
[0094] In this embodiment, the difference words can be specifically understood as words in the target sentence text that are different from the words in the corresponding position in the sentence to be expanded; the non-difference words can be specifically understood as words in the target sentence text that are the same as the words in the corresponding position in the sentence to be expanded. For each target sentence text, the second word accuracy score of each word in the target sentence text is determined by steps S206-S208, in the following manner: compare the sentence to be expanded and the target sentence text, determine whether the words in the same position are the same, if they are the same, determine that the words in this position in the target sentence text are non-difference words, if they are different, determine that the words in this position are difference words. Exemplarily, the sentence to be expanded is how are you, the target sentence text is how dare you, the difference words are dare, and how and you are non-difference words.
[0095] S207. Randomly generate a second word accuracy score for the difference word according to a score range corresponding to a preset gear, wherein the second word accuracy score for the difference word is less than the corresponding first word accuracy comprehensive score.
[0096] In this embodiment, the preset gear can be gear 5, gear 4, etc., which can be set according to factors such as scoring standards. The preset gear is determined in advance, and the score range corresponding to the preset gear can be determined according to the relationship between the gear and the score range. A score value is randomly generated, and this score value is guaranteed to be within the score range, and this score value is used as the second word accuracy score of the difference word. Taking the preset gear as gear 4 as an example, the corresponding score range is 1-3, and a value between 1-3 is randomly generated as the second word accuracy score of the difference word. And it is necessary to ensure that the second word accuracy score of the difference word is less than the first word accuracy comprehensive score of the word corresponding to the difference word in the sentence to be expanded.
[0097] S208. Determine the first word accuracy comprehensive score corresponding to the non-differential word in the sentence to be expanded as the second word accuracy score of the non-differential word.
[0098] The sentence to be expanded includes a word corresponding to the non-differential word, and the first word accuracy comprehensive score corresponding to the word in the sentence to be expanded is directly determined as the second word accuracy score of the non-differential word. That is, the non-differential word is a word that has not changed, and its score can maintain the original score unchanged.
[0099] S209, calculating the sum of the comprehensive accuracy scores of the first words to obtain a first score sum, and calculating the sum of the accuracy scores of the second words to obtain a second score sum.
[0100] S210. Multiply the ratio of the second score sum to the first score sum by the first sentence accuracy comprehensive score to obtain a second sentence accuracy comprehensive score.
[0101] For example, for the sentence to be expanded: how are you, the first sentence accuracy comprehensive score and the first word accuracy comprehensive score are: [S i 】,
w i,1 , w i,2 , w i,3
[0102] Target sentence text: how about you, then the second sentence accuracy score and the second word accuracy score are:
[0103]
[0104] in,
[0105] According to the basic principles of Table 2, the preset gear is set to 4 gears. Randomly set to 1-3 points, and need to meet
[0106] It should be noted that the second word accuracy score determined in the embodiment of the present application is a comprehensive score. The second sentence accuracy comprehensive score and the second word accuracy score can be collectively referred to as the second score information of the target sentence text.
[0107] Optionally, the score set also includes a first sentence completion score.
[0108] S211 . Determine a first sentence completeness comprehensive score according to the first sentence completeness score of the sentence to be expanded and score level information.
[0109] In this embodiment, the first sentence completeness score can be specifically understood as the score of the sentence completeness of the sentence to be expanded. The number of first sentence completeness scores can be one or more, and each score is obtained by scoring by different experts; the first sentence completeness composite score can be specifically understood as the composite score of the sentence completeness obtained by scoring by multiple experts of the sentence to be expanded.
[0110] When the expert scores the sentence to be expanded, the expert scores the completeness of the sentence to obtain a first sentence completeness score, which is saved in the corresponding score set in the first score information. When obtaining the first score information of the sentence to be expanded, the first sentence completeness score corresponding to the sentence to be expanded is obtained. The gear of the first sentence completeness comprehensive score is determined according to the score gear information, and then the comprehensive score is calculated according to the gear.
[0111] It should be noted that, when the embodiment of the present application expands the sentence to be expanded, the change in word scores may affect the completeness of the sentence. Therefore, the embodiment of the present application further calculates the completeness score of the target sentence text.
[0112] As an optional embodiment of this embodiment, this optional embodiment further determines the first sentence completeness comprehensive score according to the first sentence completeness score of the sentence to be expanded and the score gear information, and optimizes it to:
[0113] D1. Determine the completeness span information of the sentence to be expanded according to the completeness scores of the first sentences corresponding to the sentence to be expanded and the score span information.
[0114] In this embodiment, the accuracy cross-level information can be specifically understood as the level difference information of different levels of accuracy of the sentence to be expanded when it is scored by different experts.
[0115] Specifically, all first sentence completeness scores corresponding to the sentence to be expanded are determined, each first sentence completeness score is scored by an expert, score level information is determined in advance, the level where the first sentence completeness score is located is determined according to the score level information, and statistical analysis is performed on each level to determine the completeness cross-level information.
[0116] As an optional embodiment of this embodiment, this optional embodiment further determines the completeness span information of the sentence to be expanded according to the completeness scores of each first sentence corresponding to the sentence to be expanded and the score span information, and optimizes it as follows:
[0117] E1. Determine the completeness level corresponding to each first sentence completeness score according to the score level information.
[0118] In this embodiment, the completeness level can be specifically understood as the level to which the completeness score belongs. For each first sentence completeness score, the completeness score is compared with the score level information to determine the level that matches the first sentence completeness score and use it as the completeness level.
[0119] E2. Determine whether the integrity gears are the same gear. If so, determine the integrity span information as the second preset value; otherwise, determine the lowest gear and the highest gear, and add the gear difference between the lowest gear and the highest gear to the second preset value to determine the integrity span information.
[0120] In this embodiment, the second preset value may be the same as or different from the first preset value. The second preset value is set in advance to determine whether each integrity gear is the same gear. If so, the integrity span information is determined to be the second preset value. If not the same gear, all integrity gears are compared to determine the lowest gear and the highest gear among all integrity gears, and the gear difference is calculated. For example, the gears decrease from 1 to 5, the lowest gear is 4, the highest gear is 3, the gear difference is 4-3=1, the first preset value is 1, and the integrity span information is 2.
[0121] As an optional embodiment of the present embodiment, the further optimization of the present optional embodiment includes that after determining the completeness level corresponding to each first sentence completeness score according to the score level information, if the completeness level corresponding to the sentence to be expanded meets the second filtering condition, the sentence to be expanded is eliminated.
[0122] In this embodiment, the second filtering condition can be specifically understood as a condition for filtering the sentences to be expanded by the completeness level, which can be the same as or different from the first filtering condition. For example, when the completeness levels are distributed in different levels, it is determined that the second filtering condition is met; or, when the completeness levels cover all levels, the second filtering condition is met. After determining the completeness levels corresponding to the sentences to be expanded, it is determined whether the completeness levels meet the second filtering condition. If so, the sample is considered invalid and the sentences to be expanded are eliminated.
[0123] D2. Determine the comprehensive score of the first sentence completeness based on the completeness cross-file information and the completeness scores of each first sentence.
[0124] Predetermine the comprehensive score calculation method corresponding to different completeness cross-file information, determine the calculation method according to the completeness cross-file information after confirming the completeness cross-file information, calculate the completeness score of each first sentence according to the corresponding calculation method, and obtain the first sentence completeness comprehensive score. The calculation method can be to calculate the average value, maximum value, minimum value, weighted sum, etc.
[0125] As an optional embodiment of this embodiment, this optional embodiment further determines the first sentence completeness comprehensive score according to the completeness cross-file information combined with the completeness scores of each first sentence, and optimizes it to:
[0126] E1. When the completeness cross-file information is within a third preset range, an average value of the first sentence completeness scores is determined as the first sentence completeness comprehensive score.
[0127] In this embodiment, the third preset range may be the same as the first preset range, or may be different. For example, the third preset range is M≤2, where M is the completeness span information. When the completeness span information is within the third preset range, the average of the completeness scores of the first sentences is calculated, and the average is determined as the first sentence completeness comprehensive score.
[0128] E2. When the completeness cross-file information is within a fourth preset range, filter the completeness scores of the first sentences to determine at least one candidate sentence completeness score, and determine the average of the candidate sentence completeness scores as the first sentence completeness comprehensive score.
[0129] In this embodiment, the fourth preset range may be the same as the second preset range, or may be different. Taking the same as an example, the fourth preset range may be 3≤M≤5, where M is the completeness span information. The candidate sentence completeness score can be specifically understood as an alternative score for calculating the comprehensive score. When the completeness span information is within the fourth preset range, the first sentence completeness scores are screened and filtered according to preset rules to obtain at least one candidate sentence completeness score. The preset rules may be to filter out the highest score and the lowest score. Filtering the highest score and the lowest score may be to filter out one highest score and one lowest score, or to filter out all highest scores and all lowest scores; filter out the highest and lowest scores, and so on. Calculate the average of the completeness scores of the candidate sentences, and determine this average as the first sentence completeness comprehensive score.
[0130] S212. Determine a second sentence completeness comprehensive score of the target sentence text according to the second word accuracy scores, the first word accuracy comprehensive scores, and the first sentence completeness comprehensive score.
[0131] The weight or ratio is determined according to each second word accuracy score and the first word accuracy comprehensive score, and the weight or ratio is multiplied by the first sentence integrity comprehensive score to obtain the second sentence integrity comprehensive score of the target sentence text. The weight or ratio can be determined by calculating the total score of the first word accuracy comprehensive score and the total score of the second word accuracy score, and using the ratio of the two total scores as the weight or ratio; or, respectively, calculating the ratio of the first word accuracy comprehensive score and the second word accuracy score corresponding to each word, performing average, summation, weighting and other operations on each ratio, and using the value obtained as the weight or ratio.
[0132] As an optional embodiment of this embodiment, this optional embodiment further determines the second sentence integrity comprehensive score of the target sentence text according to each second word accuracy score, each first word accuracy comprehensive score and the first sentence integrity comprehensive score, and optimizes it to:
[0133] F1. Calculate the sum of the comprehensive scores of the first word accuracy to obtain a first score sum, and calculate the sum of the accuracy scores of the second words to obtain a second score sum.
[0134] F2. Multiply the ratio of the second score sum to the first score sum by the first sentence completeness comprehensive score to obtain the second sentence completeness comprehensive score.
[0135] The calculation principle of the second sentence accuracy comprehensive score and the second sentence completeness comprehensive score in the embodiment of the present application are the same, and the specific implementation method can refer to the above description.
[0136] It should be noted that, in the process of determining the second sentence accuracy score and the second sentence completeness score of the target sentence text, if the first filter condition and the second filter condition are set at the same time to screen the target sentence text, the extended sentence can be eliminated when one of the conditions is met, or the extended sentence can be eliminated when the first filter condition and the second filter condition are met at the same time. Therefore, in the process of expanding the extended sentence and determining the score, if the extended sentence meets the first filter condition and / or the second filter condition, the extended sentence can be eliminated. At this time, since the sample is invalid, it needs to be eliminated, and there is no need to expand it to generate the target sentence text. That is, the filtering in the embodiment of the present application can be performed before generating the target sentence text, to avoid the increase in workload caused by the invalid sample being expanded and then eliminated.
[0137] The embodiment of the present invention provides a method for determining the score of an expanded text, which solves the problem that the expanded text cannot automatically determine the score. The first word accuracy score, the first sentence accuracy score, and the first sentence integrity score of the extended sentence are comprehensively analyzed through the score gear information, and the corresponding cross-gear information is determined, and then the comprehensive score is determined according to the cross-gear information, the score error is reduced, and the accuracy of the score of the sentence to be expanded is guaranteed; the extended sentence is expanded to generate a target sentence text, and the comprehensive score of the sentence to be expanded is determined to determine the second sentence accuracy comprehensive score, the second integrity comprehensive score, and the second word accuracy score of the target sentence text, so as to realize the automatic determination of the score of the expanded text, without manual scoring, saving human resources, and improving efficiency, and the present application determines the comprehensive score of the accuracy of sentences and words, which is more accurate. Since the embodiment of the present application can quickly and accurately score the target sentence text, a large number of target sentence texts and corresponding scores can be generated according to demand, which greatly improves the scoring speed of the text.
[0138] Embodiment 3
[0139] Figure 3 This is a schematic diagram of the structure of a device for determining a score of an expanded text provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a first score information acquisition module 31, an accuracy comprehensive score module 32, a target sentence generation module 33 and a target sentence score determination module 34.
[0140] The first score information acquisition module 31 is used to acquire the sentence to be expanded and the first score information of the sentence to be expanded, wherein the first score information includes at least one score set, and the score set includes a first sentence accuracy score and at least one first word accuracy score;
[0141] An accuracy comprehensive score module 32, used to determine a first sentence accuracy comprehensive score and a first word accuracy comprehensive score according to the first score information and score gear information of the sentence to be expanded;
[0142] A target sentence generating module 33, used for expanding the sentence to be expanded to generate at least one target sentence text;
[0143] The target sentence score determination module 34 is used to determine, for each target sentence text, at least one second word accuracy score corresponding to the target sentence text, and determine the second sentence accuracy comprehensive score of the target sentence text based on each of the second word accuracy scores, each of the first word accuracy comprehensive scores and the first sentence accuracy comprehensive score.
[0144] The embodiment of the present invention provides a device for determining the score of an expanded text, which solves the problem that the score of the expanded text cannot be automatically determined. The first score information of the sentence to be expanded is analyzed and processed in combination with the score gear information, and the first sentence accuracy comprehensive score and the first word accuracy comprehensive score are determined to ensure the accuracy of the score of the sentence to be expanded; the sentence to be expanded is expanded to generate a target sentence text, and the second word accuracy score of each word in the target sentence text is determined. The second sentence accuracy comprehensive score of the target sentence text is determined by the second word accuracy score, the first word accuracy comprehensive score and the first sentence accuracy comprehensive score, so that the score of the expanded text is automatically determined without manual scoring, saving human resources and improving efficiency. In addition, the application determines the comprehensive score of the accuracy of sentences and words, which is more accurate.
[0145] Optionally, the accuracy comprehensive score module 32 includes:
[0146] An accuracy span information determination unit, configured to determine the accuracy span information of the sentence to be expanded according to the accuracy scores of the first sentences corresponding to the sentence to be expanded and the score span information;
[0147] A first comprehensive score determination unit, configured to determine a first sentence accuracy comprehensive score according to the accuracy cross-file information in combination with each of the first sentence accuracy scores;
[0148] The second comprehensive score determination unit is used to determine the first word accuracy comprehensive score for each word in the sentence to be expanded according to the accuracy cross-file information combined with the first word accuracy scores corresponding to the word.
[0149] Optionally, an accuracy span information determination unit is specifically used to determine the accuracy span corresponding to the accuracy score of each of the first sentences according to the score span information; determine whether each of the accuracy spans is the same span, and if so, determine the accuracy span information as a first preset value; otherwise, determine the lowest gear and the highest gear, and add the gear difference between the lowest gear and the highest gear plus the first preset value to determine the accuracy span information.
[0150] Optionally, the device further comprises:
[0151] The first filtering judgment module is used to remove the sentence to be expanded if the accuracy level corresponding to the sentence to be expanded meets the first filtering condition.
[0152] Optionally, the first comprehensive score determination unit is specifically used to determine the average value of the first sentence accuracy scores as the first sentence accuracy comprehensive score when the accuracy span information is within a first preset range; when the accuracy span information is within a second preset range, filter the first sentence accuracy scores to determine at least one alternative sentence accuracy score, and determine the average value of the alternative sentence accuracy scores as the first sentence accuracy comprehensive score.
[0153] Optionally, the second comprehensive score determination unit is specifically used to determine the average of the first word accuracy scores corresponding to the words as the first word accuracy comprehensive score when the accuracy span information is within a first preset range; when the accuracy span information is within a second preset range, filter the first word accuracy scores corresponding to the words, determine at least one alternative word accuracy score, and determine the average of the alternative word accuracy scores as the first word accuracy comprehensive score.
[0154] Optionally, the target sentence score determination module 34 includes:
[0155] A sentence comparison unit, used for comparing the sentence to be expanded with the target sentence text, and determining the difference words and non-difference words in the target sentence text;
[0156] A difference word second score determination unit, used to randomly generate a second word accuracy score of the difference word according to a score range corresponding to a preset gear, wherein the second word accuracy score of the difference word is less than the corresponding first word accuracy comprehensive score;
[0157] The non-word second score determination unit is used to determine the first word accuracy comprehensive score corresponding to the non-differential word in the sentence to be expanded as the second word accuracy score of the non-differential word.
[0158] Optionally, the target sentence score determination module 34 includes:
[0159] A first score sum calculation unit, used for calculating the sum of the first word accuracy comprehensive scores to obtain a first score sum, and calculating the sum of the second word accuracy scores to obtain a second score sum;
[0160] The second accuracy comprehensive score determination unit is used to multiply the ratio of the second score sum to the first score sum by the first sentence accuracy comprehensive score to obtain the second sentence accuracy comprehensive score.
[0161] Optionally, the score set also includes a first sentence completeness score;
[0162] The device also includes:
[0163] A first completeness comprehensive score determination module, configured to determine a first sentence completeness comprehensive score according to the first sentence completeness score of the sentence to be expanded and score gear information;
[0164] The second completeness comprehensive score determination module is used to determine the second sentence completeness comprehensive score of the target sentence text according to each of the second word accuracy scores, each of the first word accuracy comprehensive scores and the first sentence completeness comprehensive score.
[0165] Optionally, the first completeness comprehensive score determination module includes:
[0166] A completeness span information determination unit, configured to determine the completeness span information of the sentence to be expanded according to the completeness scores of each of the first sentences corresponding to the sentence to be expanded and the score span information;
[0167] A sentence completeness score determination unit is used to determine a first sentence completeness comprehensive score according to the completeness cross-file information in combination with each of the first sentence completeness scores.
[0168] Optionally, a completeness span information determination unit is used to determine the completeness span corresponding to the completeness score of each of the first sentences according to the score span information; determine whether the completeness spans are the same span, and if so, determine the completeness span information as a second preset value; otherwise, determine the lowest gear and the highest gear, and add the gear difference between the lowest gear and the highest gear plus the second preset value to determine the completeness span information.
[0169] Optionally, the device further comprises:
[0170] The second filtering judgment module is used to remove the sentence to be expanded if the completeness level corresponding to the sentence to be expanded meets the second filtering condition.
[0171] Optionally, the sentence completeness score determination unit is specifically used to determine the average value of the first sentence completeness scores as the first sentence completeness comprehensive score when the completeness cross-file information is within a third preset range; when the completeness cross-file information is within a fourth preset range, filter the first sentence completeness scores to determine at least one alternative sentence completeness score, and determine the average value of the alternative sentence completeness scores as the first sentence completeness comprehensive score.
[0172] Optionally, the second completeness comprehensive score determination module includes:
[0173] A second score sum calculation unit, used for calculating the sum of the first word accuracy comprehensive scores to obtain a first score sum, and calculating the sum of the second word accuracy scores to obtain a second score sum;
[0174] The completeness comprehensive score calculation unit is used to multiply the ratio of the second score sum to the first score sum by the first sentence completeness comprehensive score to obtain a second sentence completeness comprehensive score.
[0175] The device for determining the score of an expanded text provided in the embodiment of the present invention can execute the method for determining the score of an expanded text provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.
[0176] Embodiment 4
[0177] Figure 4 A block diagram of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as desktop computers, personal digital assistants, servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as smart phones and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.
[0178] like Figure 4As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.
[0179] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0180] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 performs the various methods and processes described above, such as the score determination method of the expanded text.
[0181] In some embodiments, the score determination method for the expanded text may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the score determination method for the expanded text described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to perform the score determination method for the expanded text in any other suitable manner (e.g., by means of firmware).
[0182] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0183] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0184] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0185] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0186] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0187] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0188] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0189] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for determining a score of an expanded text, characterized in that: include: Acquire a sentence to be expanded and first score information of the sentence to be expanded, wherein the first score information includes at least one score set, and the score set includes a first sentence accuracy score and at least one first word accuracy score; Determine a first sentence accuracy comprehensive score and a first word accuracy comprehensive score according to the first score information and the score gear information of the sentence to be expanded; Expanding the sentence to be expanded to generate at least one target sentence text; For each target sentence text, determine at least one second word accuracy score corresponding to the target sentence text, and determine the second sentence accuracy composite score of the target sentence text based on each of the second word accuracy scores, each of the first word accuracy composite scores, and the first sentence accuracy composite score.
2. The method according to claim 1, characterized in that: The step of determining the first sentence accuracy comprehensive score and the first word accuracy comprehensive score according to the first score information and the score gear information of the sentence to be expanded includes: Determine the accuracy span information of the sentence to be expanded according to the accuracy scores of the first sentences corresponding to the sentence to be expanded and the score span information; Determine a first sentence accuracy composite score based on the accuracy cross-file information combined with each of the first sentence accuracy scores; For each word in the sentence to be expanded, a comprehensive first word accuracy score is determined according to the accuracy cross-file information in combination with each first word accuracy score corresponding to the word.
3. The method according to claim 2, characterized in that The step of determining the accuracy span information of the sentence to be expanded according to the accuracy scores of the first sentences corresponding to the sentence to be expanded and the score span information includes: Determine the accuracy level corresponding to each of the first sentence accuracy scores according to the score level information; Determining whether the accuracy levels are the same level, and if so, determining the accuracy span information to be a first preset value; Otherwise, the lowest gear and the highest gear are determined, and the gear difference between the lowest gear and the highest gear is added to a first preset value to be determined as the accuracy cross-gear information.
4. The method according to claim 3, characterized in that After determining the accuracy level corresponding to each of the first sentence accuracy scores according to the score level information, the method further includes: If the accuracy level corresponding to the sentence to be expanded meets the first filtering condition, the sentence to be expanded is eliminated.
5. The method according to claim 2, characterized in that: The determining the first sentence accuracy comprehensive score according to the accuracy cross-file information combined with each of the first sentence accuracy scores includes: When the accuracy span information is within a first preset range, an average of the first sentence accuracy scores is determined as a first sentence accuracy composite score; When the accuracy span information is within a second preset range, the first sentence accuracy scores are filtered to determine at least one candidate sentence accuracy score, and the average of the candidate sentence accuracy scores is determined as the first sentence comprehensive accuracy score.
6. The method according to claim 2, characterized in that The step of determining the first word accuracy comprehensive score according to the accuracy span information combined with the first word accuracy scores corresponding to the words includes: When the accuracy span information is within a first preset range, an average of the first word accuracy scores corresponding to the words is determined as the first word accuracy comprehensive score; When the accuracy span information is within a second preset range, the first word accuracy scores corresponding to the words are filtered to determine at least one candidate word accuracy score, and the average of the candidate word accuracy scores is determined as the first word comprehensive accuracy score.
7. The method according to claim 1, characterized in that The determining of at least one second word accuracy score corresponding to the target sentence text comprises: Compare the text of the sentence to be expanded with the target sentence, and determine the difference words and non-difference words in the target sentence; Randomly generating a second word accuracy score of the difference word according to a score range corresponding to a preset gear, wherein the second word accuracy score of the difference word is less than the corresponding first word accuracy comprehensive score; The comprehensive score of the first word accuracy corresponding to the non-differential word in the sentence to be expanded is determined as the second word accuracy score of the non-differential word.
8. The method according to claim 1, characterized in that The step of determining the second sentence accuracy comprehensive score of the target sentence text according to each of the second word accuracy scores, each of the first word accuracy comprehensive scores, and the first sentence accuracy comprehensive score comprises: Calculate the sum of the first word accuracy comprehensive scores to obtain a first score sum, and calculate the sum of the second word accuracy scores to obtain a second score sum; The ratio of the second score sum to the first score sum is multiplied by the first sentence accuracy comprehensive score to obtain the second sentence accuracy comprehensive score.
9. The method according to claim 1, characterized in that: The score set also includes a first sentence completion score, and the method further includes: Determine a first sentence completeness comprehensive score according to the first sentence completeness score of the sentence to be expanded and the score gear information; The second sentence completeness comprehensive score of the target sentence text is determined according to each of the second word accuracy scores, each of the first word accuracy comprehensive scores and the first sentence completeness comprehensive score.
10. The method according to claim 9, characterized in that The step of determining the first sentence completeness comprehensive score according to the first sentence completeness score of the sentence to be expanded and the score gear information includes: Determine the completeness span information of the sentence to be expanded according to the first sentence completeness scores and score span information corresponding to the sentence to be expanded; The first sentence completeness comprehensive score is determined according to the completeness cross-file information combined with each of the first sentence completeness scores.
11. The method according to claim 10, characterized in that The determining the completeness span information of the sentence to be expanded according to the first sentence completeness scores and score span information corresponding to the sentence to be expanded includes: Determine the completeness level corresponding to each of the first sentence completeness scores according to the score level information; Determining whether the integrity levels are the same level, and if so, determining that the integrity span information is a second preset value; Otherwise, the lowest gear and the highest gear are determined, and the gear difference between the lowest gear and the highest gear is added to a second preset value to be determined as the completeness cross-gear information.
12. The method according to claim 11, characterized in that After determining the completeness level corresponding to each of the first sentence completeness scores according to the score level information, the method further includes: If the completeness level corresponding to the sentence to be expanded meets the second filtering condition, the sentence to be expanded is eliminated.
13. The method according to claim 10, characterized in that The determining the first sentence completeness comprehensive score according to the completeness cross-file information combined with each of the first sentence completeness scores includes: When the completeness span information is within a third preset range, an average of the first sentence completeness scores is determined as the first sentence completeness comprehensive score; When the completeness span information is within a fourth preset range, the first sentence completeness scores are filtered to determine at least one candidate sentence completeness score, and an average value of the candidate sentence completeness scores is determined as the first sentence completeness comprehensive score.
14. The method according to claim 9, characterized in that The step of determining the second sentence integrity comprehensive score of the target sentence text according to each of the second word accuracy scores, each of the first word accuracy comprehensive scores, and the first sentence integrity comprehensive score comprises: Calculate the sum of the first word accuracy comprehensive scores to obtain a first score sum, and calculate the sum of the second word accuracy scores to obtain a second score sum; The ratio of the second score sum to the first score sum is multiplied by the first sentence completeness comprehensive score to obtain the second sentence completeness comprehensive score.
15. A device for determining a score of an expanded text, characterized in that: include: A first score information acquisition module, used to acquire a sentence to be expanded and first score information of the sentence to be expanded, wherein the first score information includes at least one score set, and the score set includes a first sentence accuracy score and at least one first word accuracy score; An accuracy comprehensive score module, used to determine a first sentence accuracy comprehensive score and a first word accuracy comprehensive score according to the first score information and score gear information of the sentence to be expanded; A target sentence generating module, used for expanding the sentence to be expanded to generate at least one target sentence text; The target sentence score determination module is used to determine, for each target sentence text, at least one second word accuracy score corresponding to the target sentence text, and determine the second sentence accuracy comprehensive score of the target sentence text based on each of the second word accuracy scores, each of the first word accuracy comprehensive scores and the first sentence accuracy comprehensive score.
16. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so as to enable the at least one processor to perform the method for determining a score of an expanded text according to any one of claims 1 to 14.
17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining a score of an expanded text according to any one of claims 1 to 14 when executed.
Citation Information
Patent Citations
Corpus expansion method and device of speech recognition model, electronic equipment and computer readable storage medium
CN113889094A
Quality assessment method for automatic annotation of speech data
US20220164531A1
Method and apparatus for improved speech recognition by modifying a pronunciation dictionary based on pattern definitions of alternate word pronunciations
US6389394B1
