An evaluation system and method for formula reading quality of a speech model

Abstract

The application belongs to the technical field of speech signal processing, and particularly relates to an evaluation system and method for formula reading quality of a speech model. The system comprises: a standard corpus construction module, configured to obtain a standard corpus set based on formula text and a formula parser; an output acquisition module, configured to obtain predicted triplets based on the formula text and a to-be-tested speech model; a multi-dimensional evaluation module, configured to obtain a comprehensive evaluation score based on the standard corpus set and the predicted triplets; and an application feedback module, configured to determine whether the to-be-tested speech model is qualified in a formula reading task based on the comprehensive evaluation score and a preset threshold. The application can not only guarantee basic character-level accuracy, but also identify errors at the semantic level, so as to more truly reflect the reading quality of the speech model and solve the defects that the prior art cannot accurately measure formula reading accuracy and semantic consistency.

Description

Technical Field

[0001] This invention belongs to the field of speech signal processing technology, specifically relating to an evaluation system and method for the quality of speech model formula reading. Background Technology

[0002] With the rapid development of artificial intelligence and speech processing technologies, speech models have been widely used in fields such as speech recognition, speech synthesis, and human-computer interaction. In educational settings, teachers and learners not only need natural language reading and assessment, but also the accurate expression and understanding of speech presentations containing scientific content such as mathematical formulas, physical equations, and chemical symbols.

[0003] Formula reading aloud refers to the transformation of symbolic formulas into standard spoken language, enabling listeners to understand the content and structure of the formulas through audio. This ability is invaluable for classroom instruction, accessible education, and intelligent tutoring systems in online learning platforms. However, formula reading aloud is far more complex than ordinary text reading, as it involves hierarchical relationships of symbolic structures, subscripts and superscripts, operational priorities, and interdisciplinary terminology. Existing speech quality assessment methods are insufficient to meet the specific needs of formula reading aloud scenarios.

[0004] Existing speech quality assessment methods have significant shortcomings in formula reading scenarios. Traditional character-level metrics (such as word error rate (WER) and BLEU scores) can only measure text similarity and are insufficient to identify errors in symbols, units, or formula structure. While semantic assessment methods based on general language models can provide some semantic judgment, they are not optimized for the specific characteristics of formula reading, easily resulting in correct character alignment but incorrect formula meaning. Furthermore, existing methods lack interdisciplinary coverage and are difficult to apply simultaneously to assessing the quality of formula reading across multiple fields such as mathematics, physics, and chemistry. In practical teaching applications, the reading generated by speech models not only needs to be natural and fluent but also needs to accurately convey the symbols and semantics of the formula, a requirement that existing assessment methods cannot comprehensively measure.

[0005] Therefore, there is an urgent need to propose a multi-dimensional evaluation system and method for formula reading scenarios, which can not only ensure basic character-level accuracy, but also identify semantic errors, thereby more realistically reflecting the reading quality of the speech model. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides an evaluation system and method for the quality of formula reading aloud based on speech models, aiming to solve the shortcomings of existing technologies in accurately measuring the accuracy and semantic consistency of formula reading aloud.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] An evaluation system for the quality of formula reading aloud in speech models, the system comprising: a standard corpus construction module, an output acquisition module, a multi-dimensional evaluation module, and an application feedback module;

[0009] The standard corpus construction module is used to obtain a standard corpus set based on formula text and formula parser;

[0010] The output acquisition module is used to obtain the predicted triples based on the formula text and the speech model under test;

[0011] A multi-dimensional evaluation module is used to obtain a comprehensive evaluation score based on a standard corpus set and predicted triples;

[0012] The application feedback module is used to determine whether the speech model under test is qualified in the formula reading task based on the comprehensive evaluation score and preset threshold.

[0013] Preferably, the standard corpus construction module includes: a formula parsing unit, a proofreading unit, and a speech synthesis unit;

[0014] The formula parsing unit is used to convert formula text into a sequence of symbols based on the formula parser;

[0015] The proofreading unit is used to proofread the symbol sequence using a preset rule base to obtain the standard reading text;

[0016] The speech synthesis unit is used to obtain synthesized speech based on standard read-out text and a high-quality speech synthesis model.

[0017] Preferably, the output acquisition module includes: a speech generation unit and a speech recognition unit;

[0018] The speech generation unit is used to input the formula text into the speech model to be tested and obtain the predicted speech;

[0019] The speech recognition unit is used to transcribe predicted speech into predicted text based on the ASR model.

[0020] Preferably, the multi-dimensional evaluation module includes: a symbol-sensitive weighting unit, a structural consistency unit, and an interdisciplinary rule checking unit;

[0021] The symbol-sensitive weighted unit is used to obtain the symbol-weighted word error rate based on the symbol-sensitive weighted edit distance;

[0022] The structural consistency unit is used to convert formula text into a symbol tree, and obtain the difference between the standard and the prediction in the hierarchical structure based on the symbol tree;

[0023] The interdisciplinary rule checking unit is used to obtain the penalty items for interdisciplinary rule detection based on a preset rule base.

[0024] Preferably, the method for obtaining the symbol-weighted word error rate based on the symbol-weighted edit distance by the symbol-sensitive weighting unit includes:

[0025] ;

[0026] in, For symbol-weighted word error rate, For the first in the formula text The weight of each symbol is greater than that of ordinary text. For the cost function of editing operations, The length of the formula text. For standard reading text, To predict text, For the first The standard reading text of each symbol, For the first Predicted text for each symbol.

[0027] Preferably, the structural consistency unit converts the formula text into a symbol tree, and the method for obtaining the difference between the standard and the prediction in the hierarchical structure based on the symbol tree includes:

[0028] ;

[0029] in, The difference between the standard and the prediction in the hierarchical structure, For standard symbol trees, For predicting symbolic trees, This is the formula text.

[0030] Preferably, the method by which the interdisciplinary rule checking unit obtains penalty items for interdisciplinary rule detection based on a preset rule base includes:

[0031] ;

[0032] in, As a penalty item, For rule weights, For indicator functions that violate the conditions, For the rule base The rules in This indicates a violation of the rules.

[0033] Preferably, the multi-dimensional evaluation module obtains a comprehensive evaluation score based on a standard corpus set and predicted triples, including the following methods:

[0034] ;

[0035] in, To comprehensively evaluate the scores, For a standard triplet, To predict triples, , , These are all weighted coefficients, controlling the contribution ratio of different dimensions in the overall evaluation score.

[0036] This invention also provides an evaluation method for the quality of formula reading aloud based on a speech model. The method is implemented using the aforementioned system and includes:

[0037] Based on formula text and formula parser, a standard corpus is obtained;

[0038] Based on the formula text and the speech model to be tested, the predicted triplet is obtained;

[0039] A comprehensive evaluation score is obtained based on a standard corpus and predicted triples.

[0040] Based on the comprehensive evaluation score and the preset threshold, it is determined whether the speech model under test is qualified in the formula reading task.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] (1) This invention is specifically designed for formula reading tasks. It introduces symbol-sensitive weighting and structural consistency checking mechanisms, which can effectively identify problems such as missing symbols, incorrect subscripts and superscripts, and misjudgment of operation priority, thus overcoming the shortcomings of traditional character-level indicators that cannot capture formula details.

[0043] (2) This invention proposes a multi-dimensional fusion evaluation framework that combines character-level precision with structured semantic verification, which ensures both the accuracy of basic transcription and the correct transmission of the overall semantics of the formula, thereby achieving a comprehensive measurement of the reading quality of the speech model.

[0044] (3) The present invention designs an interdisciplinary rule verification module, which establishes a rule base for the requirements of formula reading in mathematics, physics, chemistry and other fields. It can evaluate the performance of multidisciplinary speech models under a unified framework and has good versatility and scalability.

[0045] (4) By introducing structured comparison and rule constraints, this invention can identify the situation of “characters are correctly aligned but semantically incorrect”, making the evaluation results more objective and reliable, and significantly improving the discrimination power and credibility of the evaluation system.

[0046] (5) The evaluation system and method provided by the present invention can be directly applied to intelligent education and voice interaction scenarios, forming a unified standard for formula reading. It can be used for quantitative comparison of model effects in teaching, and can also serve as an important reference for iterative optimization of large voice models. It has high engineering practical value and promotion prospects. Attached Figure Description

[0047] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of a system module for evaluating the quality of speech model formula reading in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] Example 1

[0052] like Figure 1 As shown, the present invention provides an evaluation system for the quality of speech model formula reading, including: a standard corpus construction module, an output acquisition module, a multi-dimensional evaluation module, and an application feedback module;

[0053] The standard corpus construction module is used to obtain a standard corpus set based on formula text and formula parser;

[0054] The output acquisition module is used to obtain the predicted triples based on the formula text and the speech model under test;

[0055] A multi-dimensional evaluation module is used to obtain a comprehensive evaluation score based on a standard corpus set and predicted triples;

[0056] The application feedback module is used to determine whether the speech model under test is qualified in the formula reading task based on the comprehensive evaluation score and preset threshold.

[0057] Furthermore, the specific implementation process of this invention is as follows:

[0058] The standard corpus construction module includes: a formula parsing unit, a proofreading unit, and a speech synthesis unit;

[0059] The formula parsing unit is used to convert formula text into a sequence of symbols based on the formula parser;

[0060] The proofreading unit is used to proofread the symbol sequence using a preset rule base to obtain the standard reading text;

[0061] The speech synthesis unit is used to obtain synthesized speech based on standard read-out text and a high-quality speech synthesis model.

[0062] Specifically, the input to the standard corpus construction module is multidisciplinary formula texts. This includes fields such as mathematics, physics, and chemistry. First, a formula parser is used to... Convert to a symbol sequence, for example: Parsed as " x Superscript 2 plus y The superscript 2 equals z The superscript 2" is then used. The symbol sequence is then checked against a pre-defined rule base (each generated symbol sequence is compared against the rule base to check if the symbol pronunciation, superscript / superscript structure, and order of operations conform to the standard. When inconsistencies are detected, they are replaced or adjusted according to the corresponding rules, outputting a standardized symbol sequence that conforms to the standard pronunciation, providing accurate input for subsequent speech synthesis), resulting in the standard reading text. Finally, a high-quality speech synthesis model (such as the CosyVoice2 model; this invention does not impose specific restrictions on the high-quality speech synthesis model) is used to generate the corresponding synthesized speech. The resulting paired triplet for:

[0063] ;

[0064] The final standard corpus is , n This refers to the standard corpus size.

[0065] Furthermore, the output acquisition module includes: a speech generation unit and a speech recognition unit;

[0066] The speech generation unit is used to input the formula text into the speech model to be tested and obtain the predicted speech;

[0067] The speech recognition unit is used to transcribe predicted speech into predicted text based on the ASR model.

[0068] Specifically, the input to the output acquisition module is the formula text. and the speech model to be tested. The output acquisition module will... Input the speech model to be tested and generate the predicted speech. And using an automatic speech recognition (ASR) model to Transcription to predictive text Thus forming the prediction triplet :

[0069] .

[0070] Furthermore, the multi-dimensional evaluation module includes: a symbol-sensitive weighted unit, a structural consistency unit, and an interdisciplinary rule checking unit;

[0071] The symbol-sensitive weighted unit is used to obtain the symbol-weighted word error rate based on the symbol-sensitive weighted edit distance;

[0072] The structural consistency unit is used to convert formula text into a symbol tree, and obtain the difference between the standard and the prediction in the hierarchical structure based on the symbol tree;

[0073] The interdisciplinary rule checking unit is used to obtain the penalty items for interdisciplinary rule detection based on a preset rule base.

[0074] The multi-dimensional assessment module combines symbol-sensitive weighting, structural consistency, and interdisciplinary rule checks to comprehensively improve the accuracy and applicability of the assessment results.

[0075] Specifically, the symbol-sensitive weighted unit introduces symbol-sensitive weighted edit distance. Considering that operators, variables, subscripts, and superscripts in formula text are more critical than ordinary text, the symbol-weighted word error rate is defined as:

[0076] ;

[0077] in, For symbol-weighted word error rate, For the first in the formula text The weight of each symbol (operator, variable, subscript, superscript), with symbols and units having a greater weight than ordinary text; For the cost function of editing operations, The length of the formula text. For the first The standard reading text of each symbol, For the first The indicator predicts text based on the number of symbols. It is more sensitive to key symbol errors, ensuring that critical errors in formula symbols and subscripts / superscripts are highlighted.

[0078] The structural consistency unit transforms the formula text into a symbol tree. The methods for obtaining the differences in hierarchical structure between the standard and the prediction based on the symbol tree specifically include:

[0079] First, read the formula text. The input formula text is converted into a symbol tree. Specifically, the input formula text can be converted into a MathML syntax tree using regular expressions. The formula parser establishes a hierarchical relationship based on the precedence rules of symbols, with operators as parent nodes and operands as child nodes. For example, the formula... It can be parsed as a tree, where the root node is a "plus sign" and the left subtree is a "+ sign". x The square of", the right subtree is " y The resulting symbol tree is denoted as the standard symbol tree. And predicting text The prediction symbol tree is obtained through the same process. Then, the difference between the tree edit distance (TED) metric and the prediction in terms of hierarchical structure is calculated, and nested brackets, subscripts / superscripts, and incorrect order of operations are detected.

[0080] ;

[0081] in, This indicates the difference between the standard and the prediction in the hierarchical structure.

[0082] The interdisciplinary rule checking unit obtains the penalty items for interdisciplinary rule detection based on a pre-set rule base, specifically including the following methods:

[0083] Establish rule bases for different disciplines (Similar to the rule base used when calibrating symbol sequences), for example in chemistry. It must be read as "water" or "two hydrogens and one oxygen," a unit in physics. and Do not confuse them. Perform a validity check on the symbols and units in the predicted reading results, and generate penalty items for rule violations. The penalty items are defined as follows:

[0084] ;

[0085] in, As a penalty item, For rule weights, For indicator functions that violate the conditions, For the rule base The rules in This indicates a violation of the rules.

[0086] Furthermore, the multi-dimensional evaluation module, based on a standard corpus and predicted triples, obtains a comprehensive evaluation score using the following methods:

[0087] Based on the above dimensions, the final comprehensive evaluation score is defined as follows:

[0088] ;

[0089] in, To comprehensively evaluate the scores, , , These are all weighted coefficients, controlling the contribution ratio of different dimensions in the overall evaluation score.

[0090] The multi-dimensional fusion module weights and fuses symbolic difference measure (symbol-weighted word error rate), structural difference measure (difference between standard and prediction in hierarchical structure), and interdisciplinary penalty (penalty for rule violation) to form a final comprehensive evaluation score, which is used to measure the overall quality of the speech model under test in the formula reading task.

[0091] Furthermore, the application feedback module includes a threshold decision unit. This unit determines whether the speech model under test passes the formula reading task based on the comprehensive evaluation score and a preset threshold. Specifically, it includes:

[0092] The input to the application feedback module is the comprehensive evaluation score. The application feedback module will comprehensively evaluate the score and preset thresholds. If a comparison is made, If so, the speech model under test is deemed qualified in the formula reading task; if If the test result is negative, the speech model is deemed unqualified in the formula reading task.

[0093] .

[0094] In summary, the evaluation system for the quality of formula reading aloud using a speech model provided by this invention can be embedded into online teaching platforms in educational applications. For example, in a math class, the teacher can use the speech model to read formula text aloud. The system calculates in real time. Feedback is provided. When the score is higher than a preset threshold, the reading result can be directly used for teaching demonstrations; when the score is lower than the preset threshold, a rereading is triggered or the system prompts "inaccurate reading," preventing students from receiving incorrect formula expressions. This invention achieves a complete closed loop from corpus construction, result acquisition, multi-dimensional evaluation to practical application feedback, ensuring the feasibility and practical application value of the evaluation system.

[0095] Example 2

[0096] Based on the same inventive concept, this invention also provides a method for evaluating the quality of speech model formula reading, implemented using the system described in the foregoing embodiments, the method comprising:

[0097] Based on formula text and formula parser, a standard corpus is obtained;

[0098] Based on the formula text and the speech model to be tested, the predicted triplet is obtained;

[0099] A comprehensive evaluation score is obtained based on a standard corpus and predicted triples.

[0100] Based on the comprehensive evaluation score and the preset threshold, it is determined whether the speech model under test is qualified in the formula reading task.

[0101] The specific method is as follows:

[0102] First, a formula parser is used to process formula texts from multiple disciplines. (Including fields such as mathematics, physics, and chemistry) are converted into symbol sequences, for example: Parsed as " x Superscript 2 plus y The superscript 2 equals z The superscript 2 is then used to proofread the symbol sequence using a preset rule base to obtain the standard reading text. Finally, a high-quality speech synthesis model is used to generate the corresponding synthesized speech. The resulting paired triplet for:

[0103] ;

[0104] The final standard corpus is , n This refers to the standard corpus size.

[0105] Will Input the speech model to be tested and generate the predicted speech. And using an automatic speech recognition (ASR) model to Transcription to predictive text Thus forming the prediction triplet :

[0106] .

[0107] Introducing symbol-sensitive weighted edit distance, and considering that operators, variables, subscripts, and superscripts in formula text are more critical than in ordinary text, the symbol-weighted error rate is defined as:

[0108] ;

[0109] in, For symbol-weighted word error rate, For the first in the formula text The weight of each symbol is greater than that of ordinary text. For editing operation cost function, The length of the formula text. For the first The standard reading text of each symbol, For the first The predicted text is based on a single symbol. This indicator is more sensitive to errors in key symbols.

[0110] Formula text The input formula text is converted into a symbol tree. Specifically, the input formula text can be converted into a MathML syntax tree using regular expressions. Based on the precedence rules of symbols, a hierarchical relationship is established with operators as parent nodes and operands as child nodes. For example, the formula... It can be parsed as a tree, where the root node is a "plus sign" and the left subtree is a "+ sign". x The square of", the right subtree is " y The resulting symbol tree is denoted as the standard symbol tree. And predicting text The prediction symbol tree is obtained through the same process. Then, the difference between the tree edit distance (TED) metric and the prediction in the hierarchical structure is calculated:

[0111] ;

[0112] in, This indicates the difference between the standard and the prediction in the hierarchical structure.

[0113] Establish rule bases for different disciplines For example, in chemistry It must be read as "water" or "two hydrogens and one oxygen," a unit in physics. and Do not confuse them. Define the penalty item as:

[0114] ;

[0115] in, As a penalty item, For rule weights, For indicator functions that violate the conditions, For the rule base The rules in This indicates a violation of the rules.

[0116] Based on the above dimensions, the final comprehensive evaluation score is defined as follows:

[0117] ;

[0118] in, To comprehensively evaluate the scores, , , These are all weighted coefficients, controlling the contribution ratio of different dimensions in the overall evaluation score.

[0119] The comprehensive evaluation score and the preset threshold will be combined If a comparison is made, If so, the speech model under test is deemed qualified in the formula reading task; if If the test result is negative, the speech model is deemed unqualified in the formula reading task.

[0120] .

[0121] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. An evaluation system for the quality of formula reading aloud based on speech models, characterized in that, The system includes: a standard corpus construction module, an output acquisition module, a multi-dimensional evaluation module, and an application feedback module; The standard corpus construction module is used to obtain a standard corpus set based on formula text and formula parser; The output acquisition module is used to obtain the predicted triples based on the formula text and the speech model under test; A multi-dimensional evaluation module is used to obtain a comprehensive evaluation score based on a standard corpus set and predicted triples; The application feedback module is used to determine whether the speech model under test is qualified in the formula reading task based on the comprehensive evaluation score and the preset threshold. The multi-dimensional evaluation module includes: a symbol-sensitive weighting unit, a structural consistency unit, and an interdisciplinary rule checking unit; The symbol-sensitive weighted unit is used to obtain the symbol-weighted word error rate based on the symbol-sensitive weighted edit distance; The structural consistency unit is used to convert formula text into a symbol tree, and obtain the difference between the standard and the prediction in the hierarchical structure based on the symbol tree; The interdisciplinary rule checking unit is used to obtain penalty items for interdisciplinary rule detection based on a preset rule base; The method for obtaining the symbol-weighted word error rate based on the symbol-weighted edit distance of the symbol-sensitive weighting unit includes: ； in, For symbol-weighted word error rate, For the first in the formula text The weight of each symbol is greater than that of ordinary text. For the cost function of editing operations, The length of the formula text. For standard reading text, To predict text, For the first The standard reading text of each symbol, For the first Predicted text for each symbol; The structural consistency unit transforms the formula text into a symbol tree. Methods for obtaining the differences in hierarchical structure between the standard and the prediction based on the symbol tree include: ； in, The difference between the standard and the prediction in the hierarchical structure, For standard symbol trees, For predicting symbolic trees, This is the formula text.

2. The system according to claim 1, characterized in that, The standard corpus construction module includes: a formula parsing unit, a proofreading unit, and a speech synthesis unit; The formula parsing unit is used to convert formula text into a sequence of symbols based on the formula parser; The proofreading unit is used to proofread the symbol sequence using a preset rule base to obtain the standard reading text; The speech synthesis unit is used to obtain synthesized speech based on standard read-out text and a high-quality speech synthesis model.

3. The system according to claim 1, characterized in that, The output acquisition module includes: a speech generation unit and a speech recognition unit; The speech generation unit is used to input the formula text into the speech model to be tested and obtain the predicted speech; The speech recognition unit is used to transcribe predicted speech into predicted text based on the ASR model.

4. The system according to claim 1, characterized in that, The method by which the interdisciplinary rule checking unit obtains penalty items for interdisciplinary rule detection based on a preset rule base includes: ； in, As a penalty item, For rule weights, For indicator functions that violate the conditions, For the rule base In the context of rules, "violated" indicates that a rule has been violated.

5. The system according to claim 4, characterized in that, The multi-dimensional evaluation module obtains a comprehensive evaluation score based on a standard corpus and predicted triples, including the following methods: ； in, To comprehensively evaluate the scores, For a standard triplet, To predict triples, , , These are all weighted coefficients, controlling the contribution ratio of different dimensions in the overall evaluation score.

6. A method for evaluating the quality of formula reading aloud based on a speech model, wherein the method is implemented using the system described in any one of claims 1-5, characterized in that, The method includes: Based on formula text and formula parser, a standard corpus is obtained; Based on the formula text and the speech model to be tested, the predicted triplet is obtained; A comprehensive evaluation score is obtained based on a standard corpus and predicted triples. Based on the comprehensive evaluation score and the preset threshold, it is determined whether the speech model under test is qualified in the formula reading task.

Images