Chinese accent bias error evaluation method and system based on bias error dictionary and sequence alignment

Abstract

The invention discloses a Chinese accent bias error evaluation method and system based on a bias error dictionary and sequence alignment, and relates to the technical field of text analysis, and the method comprises the following steps: collecting a voice signal of a standard phoneme text read by an evaluation object, and recognizing and generating a to-be-evaluated phoneme text; comparing the to-be-evaluated phoneme text with the standard phoneme text, and constructing an accent bias dictionary; calculating an editing distance between the to-be-evaluated phoneme text and the standard phoneme text based on the accent bias dictionary; and evaluating the accent bias of the evaluation object based on the editing distance. According to the method, a detailed accent bias dictionary is constructed, on the basis, a Levenshtein sequence alignment algorithm is used for calculating the editing distance between a to-be-evaluated text and a standard text, and then the dictionary bias editing distance and other error editing distances are counted on the basis of the dictionary; the weighted vectors of the two distances are carried out to obtain a comprehensive bias quantization score, and the score can reflect the accent bias degree of the learner more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of text analysis, and in particular to a Chinese accent error assessment method and system based on an error dictionary and sequence alignment. Background Art

[0002] In the process of learning Chinese, non-native learners often make errors in intonation and pronunciation due to factors such as the influence of their mother tongue, age, and teaching methods. Analyzing and evaluating the accent errors of non-native learners can help us gain a deeper understanding of their mastery of the target language. By analyzing specific phonetic errors, such as initial consonants, finals, and tone errors, learners can practice and overcome these difficulties in a targeted manner. In short, error analysis helps improve the quality of international Chinese teaching, promote the understanding of the second language acquisition mechanism, gain a deeper understanding of students' needs, and provide targeted improvement strategies for teaching.

[0003] Accent error assessment research currently faces many challenges. First, accent error assessment research often relies on limited data sets, which may lead to insufficient generalization of the assessment results. Second, current research focuses on a single accent or dialect, and lacks research on the comprehensive assessment of multiple accents. In addition, there is no unified assessment standard for accent errors, and different studies may use different assessment methods, which makes it difficult to compare and synthesize research. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of lack of evaluation criteria for accent errors in the prior art.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a Chinese accent error assessment method based on an error dictionary and sequence alignment, comprising the following steps:

[0006] Collect the speech signal of the evaluation object reading the standard phoneme text, and recognize and generate the phoneme text to be evaluated;

[0007] Compare the phoneme text to be evaluated with the standard phoneme text and construct an accent error dictionary;

[0008] Calculate the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary;

[0009] The accent error of the evaluation object is evaluated based on the edit distance.

[0010] Preferably, the construction of the accent error dictionary comprises the following steps:

[0011] Compare the phoneme text to be evaluated with the standard phoneme text, count the incorrectly mapped syllable pairs, and filter out the syllable pairs with a frequency higher than the preset threshold Error mapping, get the high-frequency error mapping set ,in, Indicates the wrong speech recognition phoneme, Indicates standard phonemes;

[0012] Mapping high-frequency errors to sets and the Global Accent Error Dataset Perform integration to obtain an accent error dictionary , the Global Accent Error Dataset Contains syllable substitution patterns of pronunciation errors in different countries.

[0013] Preferably, the method of calculating the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary adopts the Levenshtein sequence alignment algorithm, and includes the following steps:

[0014] The standard phoneme text is used as the A sequence, and the phoneme text to be evaluated is used as the B sequence;

[0015] Initialize the matrix and set the size of the matrix to (m+1)×(n+1), where m is the length of sequence A and n is the length of sequence B; the first row and the first column of the matrix are initialized to the sequence from 0 to n and the sequence from 0 to m respectively;

[0016] Fill the matrix, starting from i=1, j=1, and gradually fill each position (i, j) in the matrix; i∈[0, m] represents the row index, j∈[0, n] represents the column index; the value dp[i][j] filled to the position (i, j) represents the edit distance, which is determined by the comparison result of the i-th element of sequence A and the j-th element of sequence B;

[0017] After completing the filling of the matrix, take (m, n) as the backtracking point, calculate the difference between the value of the backtracking point and the minimum value in the adjacent position of the backtracking point as the distance, and use the position of the minimum value as the new backtracking point to continue calculating the distance until the position (0, 0), and then connect the distances of all backtracking points as a sequence representing the minimum editing operation path; for the backtracking point (i, j), its adjacent positions include (i-1, j), (i, j-1), (i-1, j-1);

[0018] The edit distance is calculated based on the accent error dictionary in the sequence representing the minimum editing operation path.

[0019] Preferably, the value filled into the position (i, j) is determined according to the comparison result of the i-th element of the A sequence and the j-th element of the B sequence, specifically:

[0020] if equal ,but , indicating that no operation is required;

[0021] if Not equal to ,but , indicating that editing is required; among them, Represents an insert operation. Indicates a delete operation. Indicates a replacement operation; , and Both represent operation weights, and , .

[0022] Preferably, the step of calculating the edit distance in a sequence representing a minimum edit operation path based on an accent error dictionary comprises:

[0023] Classifying the edit distances in the sequence representing the minimum edit operation path, if the edit operation corresponding to the edit distance is a replacement operation and the replacement operation conforms to the error mapping of the accent error dictionary D, then classifying the edit distance as a dictionary error edit distance; otherwise, classifying the edit distance as an error edit distance;

[0024] Classify and add the edit distances in the sequence representing the minimum edit operation path, and add all dictionary error edit distances to get the total dictionary error edit distance. , add up all the error edit distances to get the total error edit distance .

[0025] Preferably, the method of evaluating the accent error of the evaluation object based on the edit distance is specifically as follows: assigning different weights to different types of edit distances, and adding the weights of the edit distances to obtain an accent error score, wherein a higher score indicates a more serious pronunciation error of the evaluation object; the accent error score It is expressed as:

[0026] ;

[0027] in, for The weight of for The weight of .

[0028] The present invention also provides a Chinese accent error assessment system based on an error dictionary and sequence alignment, comprising:

[0029] The voice collection module collects the voice signal of the evaluation object reading the standard phoneme text, performs voice recognition to generate the phoneme text to be evaluated;

[0030] The dictionary building module compares the phoneme text to be evaluated with the standard phoneme text to build an accent error dictionary;

[0031] A distance calculation module calculates the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary;

[0032] The accent scoring module evaluates the accent error of the assessment object based on the edit distance.

[0033] The present invention has the following beneficial effects:

[0034] (1) The present invention constructs a detailed accent error dictionary. On this basis, the Levenshtein sequence alignment algorithm is used to calculate the edit distance between the phoneme text obtained by speech recognition and the standard phoneme text, and the algorithm's backtracking mechanism is used to count the edit distance of the errors in the dictionary and other incorrect edit distances; by weighting these two distances, a comprehensive error quantification score is obtained, which can more accurately reflect the degree of the learner's accent error; this method not only focuses on the applicability of oral communication for non-native Chinese learners, but also provides targeted improvement suggestions for teaching and learning through precise quantitative evaluation. Through the application of this method, the quality and effect of Chinese teaching can be significantly improved, and the learners' oral ability can be further improved.

[0035] (2) The innovation of the present invention lies in its in-depth analysis and quantitative evaluation of the error recognition process, which provides a new perspective and tool for the teaching of Chinese as a second language. Through the systematic study and quantification of errors, different editing weights are assigned to different editing operations (for phonemes, if replacement is required, it means that the difference between the two phoneme sequences is large, and the pronunciation error reflected is greater. Giving a larger editing weight to replacement can better reflect the degree of error), which helps teachers to more effectively identify and correct learners' oral errors, and also provides learners with clear learning goals and feedback, thereby promoting two-way interaction and optimization of teaching and learning.

[0036] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A method step diagram of an embodiment of the present invention;

[0038] Figure 2 4 is a system architecture diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0039] See also Figure 1 FIG. 1 is a diagram showing steps of a method according to an embodiment of the present invention, comprising the following steps:

[0040] S101, collecting a speech signal of the evaluation object reading a standard phoneme text, and performing speech recognition to generate a phoneme text to be evaluated;

[0041] S102, comparing the phoneme text to be evaluated with the standard phoneme text, and constructing an accent error dictionary;

[0042] S103, calculating the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary;

[0043] S104, evaluating the accent error of the evaluation object based on the edit distance.

[0044] Specifically, the step S102 includes the following steps:

[0045] S1021, compare the phoneme text to be evaluated with the standard phoneme text, count the incorrectly mapped syllable pairs, and filter out the syllable pairs with a frequency higher than a preset threshold Error mappings such as , get the high-frequency error mapping set ,in, Indicates the wrong speech recognition phoneme, represents the standard phoneme; the data set used in this embodiment is , N=22753 is the size of the dataset. Examples of partial syllable substitution statistics are shown in Table 1.

[0046] Table 1 Example of syllable substitution statistics:

[0047]

[0048] S1022, mapping high frequency errors to sets and the Global Accent Error Dataset Perform integration to obtain an accent error dictionary , the Global Accent Error Dataset Contains syllable substitution patterns of pronunciation errors in different countries; among them, The typical pronunciation errors of learners from different countries provided by the "Global Accent Chinese" project (https: / / www.globalaccentchinese.com / ) are used to verify the syllable substitution patterns obtained from actual speech data. The dictionary contains syllable substitution patterns of pronunciation errors in different countries, as shown in Table 2, which are examples of pronunciation errors in Thai and Vietnamese accents.

[0049] Table 2 Examples of pronunciation errors in Thai and Vietnamese accents:

[0050]

[0051] Non-native Chinese learners in Thailand often use the following substitution rules when pronouncing Chinese: h k;zh z;sh c; in terms of finals: a o;ia ie;ui ao; in tone: 5 . Non-native Chinese learners in Vietnam usually use the following substitution rules when pronouncing initial consonants: p d, b p,d t,c ch,zh z, q j,ch q; in terms of finals: e o,ou uo; in terms of tone: 4 The phonemes on the left side of the arrow represent the phonemes that are often mispronounced, while the phonemes on the right side of the arrow are the corrected phonemes.

[0052] Specifically, the S103 includes the following steps:

[0053] S1031, using the standard phoneme text as the A sequence and the phoneme text to be evaluated as the B sequence; this embodiment takes A = "shi4 jie4 de5", B = "shi4 qie4 de1" as an example for explanation;

[0054] S1032, initialize a two-dimensional matrix dp, set the size of the matrix to (m+1)×(n+1), where m is the length of sequence A and n is the length of sequence B; in this embodiment, the size of the matrix is ​​4×4, and the first row and the first column of the matrix are initialized to sequences from 0 to 3 respectively;

[0055] S1033, fill the matrix, starting from i=1, j=1, and gradually fill each position (i, j) in the matrix; the value dp[i][j] filled to the position (i, j) represents the edit distance, which is determined by the comparison result of the i-th element of sequence A and the j-th element of sequence B; i∈[0, m] represents the row index, j∈[0, n] represents the column index, but note that the filling of this step starts from i=1, j=1;

[0056] S1034, after completing the filling of the matrix, take (m, n) as the backtracking point, calculate the difference between the value of the backtracking point and the minimum value in the adjacent position of the backtracking point as the distance, and take the position of the minimum value as the new backtracking point to continue calculating the distance until the position (0, 0), and then connect the distances of all the backtracking points as a sequence representing the minimum editing operation path; for the backtracking point (i, j), its adjacent positions include (i-1, j), (i, j-1), (i-1, j-1);

[0057] S1035: Calculate edit distance according to the minimum edit operation path.

[0058] Specifically, the S1033 is as follows: if equal ,but , indicating that no action is required; if Not equal to ,but , indicating that an edit operation is required and the edit operation with the smallest edit distance is selected; there are three types of edit operations: Represents an insert operation. Indicates a delete operation. Indicates a replacement operation; , and Both represent operation weights, and , .For example, ,express The "shi4" and The "shi4" in the sequence is equal and no editing operation is required; , ,but , which means that the edit distance of replacing "shi4" with "qie4" is 1 (the edit distance of the insertion operation is 3, the edit distance of the replacement operation is 1, and the edit distance of the deletion operation is 2, so the minimum value 1 is selected as The filled matrix is ​​shown in Table 3.

[0059] Table 3 Edit distance table:

[0060]

[0061] Specifically, in S1034, backtracking starts from the lower right corner as the first backtracking point Matrix, find the sequence representing the minimum editing operation path during the backtracking process, that is, find the values ​​of the adjacent positions (top, left, top left corner) of each backtracking point in the matrix, calculate the difference between the value of the backtracking point and the minimum value among them as the distance, and use the position of the minimum value as the new backtracking point to continue calculating the distance; connect the distance of each backtracking point as the sequence of the latest operation path. In this embodiment, the backtracking point dp[3][3]=4, and the minimum value in its adjacent position is 2 (i.e., dp[2][2]). The difference between the backtracking points dp[3][3] and dp[2][2] is 2 as the distance; dp[2][2] is taken as the new backtracking point, and the minimum value in its adjacent position is 0 (i.e., dp[1][1]). The difference between the backtracking points dp[2][2] and dp[1][1] is 2 as the distance; dp[1][1] is taken as the new backtracking point, and the minimum value in its adjacent position is 0 (i.e., dp[0][0]). The difference between the backtracking points dp[1][1] and dp[0][0] is 0 as the distance; therefore, the sequence 2, 2, 0 is finally obtained, and the alignment result table based on the sequence is shown in Table 4.

[0062] Table 4 Alignment results table:

[0063]

[0064] Specifically, the dictionary error edit distance is calculated in S1035 and error edit distance Among them, the dictionary error edit distance is 2, which means "shi4" is replaced by "qie4"; Wrong edit distance is 2, which means "de1" is replaced by "de5".

[0065] Specifically, S104 is as follows: assigning different weights to different types of edit distances, adding the weights of the edit distances, and obtaining a result as an accent error score. A higher score indicates a more serious pronunciation error of the non-native speaker. for The weight of =0.7 The weight of; According to Table 4, the dictionary error edit distance is obtained 2, wrong edit distance If it is 2, the weighted quantitative evaluation score of the accent error is The calculation is as follows:

[0066]

[0067] See also Figure 2 FIG. 1 is a system structure diagram of an embodiment of the present invention, including:

[0068] The speech collection module 201 collects the speech signal of the evaluation object reading the standard phoneme text, performs speech recognition to generate the phoneme text to be evaluated;

[0069] A dictionary building module 202 compares the phoneme text to be evaluated with the standard phoneme text to build an accent error dictionary;

[0070] A distance calculation module 203 calculates the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary;

[0071] The accent scoring module 204 evaluates the accent error of the assessment object based on the edit distance.

[0072] The present invention pays full attention to the communicative applicability of spoken language of non-native Chinese learners, improves the accuracy of accent error recognition, and provides specific improvement suggestions for teaching and learning, thereby improving the quality and effect of Chinese teaching.

[0073] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A Chinese accent error assessment method based on error dictionary and sequence alignment, characterized in that: The steps include: Collect the speech signal of the evaluation object reading the standard phoneme text, and recognize and generate the phoneme text to be evaluated; Compare the phoneme text to be evaluated with the standard phoneme text and construct an accent error dictionary; Calculate the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary; The accent error of the evaluation object is evaluated based on the edit distance.

2. The Chinese accent error assessment method based on error dictionary and sequence alignment according to claim 1, characterized in that: The construction of the accent error dictionary comprises the following steps: Compare the phoneme text to be evaluated with the standard phoneme text, count the incorrectly mapped syllable pairs, and filter out the syllable pairs with a frequency higher than the preset threshold Error mapping, get the high-frequency error mapping set ,in, Indicates the wrong speech recognition phoneme, Indicates standard phonemes; Mapping high-frequency errors to sets and the Global Accent Error Dataset Perform integration to obtain an accent error dictionary , the Global Accent Error Dataset Contains syllable substitution patterns of pronunciation errors in different countries.

3. The Chinese accent error assessment method based on error dictionary and sequence alignment according to claim 2, characterized in that: The method of calculating the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary adopts the Levenshtein sequence alignment algorithm, including the following steps: The standard phoneme text is used as the A sequence, and the phoneme text to be evaluated is used as the B sequence; Initialize the matrix and set the size of the matrix to (m+1)×(n+1), where m is the length of sequence A and n is the length of sequence B; the first row and the first column of the matrix are initialized to the sequence from 0 to n and the sequence from 0 to m respectively; Fill the matrix, starting from i=1, j=1, and gradually fill each position (i, j) in the matrix; i∈[0, m] represents the row index, j∈[0, n] represents the column index; the value dp[i][j] filled to the position (i, j) represents the edit distance, which is determined by the comparison result of the i-th element of sequence A and the j-th element of sequence B; After completing the filling of the matrix, take (m, n) as the backtracking point, calculate the difference between the value of the backtracking point and the minimum value in the adjacent position of the backtracking point as the distance, and use the position of the minimum value as the new backtracking point to continue calculating the distance until the position (0, 0), and then connect the distances of all backtracking points as a sequence representing the minimum editing operation path; for the backtracking point (i, j), its adjacent positions include (i-1, j), (i, j-1), (i-1, j-1); The edit distance is calculated based on the accent error dictionary in the sequence representing the minimum editing operation path.

4. The Chinese accent error assessment method based on error dictionary and sequence alignment according to claim 3 is characterized in that: The value filled to the position (i, j) is determined according to the comparison result of the i-th element of the A sequence and the j-th element of the B sequence, specifically: if equal ,but , indicating that no operation is required; if Not equal to ,but , indicating that editing operation is required; among them, Represents an insert operation. Indicates a delete operation. Indicates a replacement operation; , and Both represent operation weights, and , .

5. The Chinese accent error assessment method based on error dictionary and sequence alignment according to claim 4, characterized in that: The method of calculating the edit distance in a sequence representing a minimum edit operation path based on an accent error dictionary includes: Classifying the edit distances in the sequence representing the minimum edit operation path, if the edit operation corresponding to the edit distance is a replacement operation and the replacement operation conforms to the error mapping of the accent error dictionary D, then classifying the edit distance as a dictionary error edit distance; otherwise, classifying the edit distance as an error edit distance; Classify and add the edit distances in the sequence representing the minimum edit operation path, and add all dictionary error edit distances to get the total dictionary error edit distance. , add up all the error edit distances to get the total error edit distance .

6. The Chinese accent error assessment method based on error dictionary and sequence alignment according to claim 4, characterized in that: The method of evaluating the accent error of the evaluation object based on the edit distance is specifically as follows: assigning different weights to different types of edit distances, and adding the weights of the edit distances to obtain an accent error score, wherein a higher score indicates a more serious pronunciation error of the evaluation object; an accent error score It is expressed as: ； in, for The weight of for The weight of .

7. A Chinese accent error assessment system based on error dictionary and sequence alignment, characterized in that: include: The voice collection module collects the voice signal of the evaluation object reading the standard phoneme text, performs voice recognition to generate the phoneme text to be evaluated; The dictionary building module compares the phoneme text to be evaluated with the standard phoneme text to build an accent error dictionary; A distance calculation module calculates the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary; The accent scoring module evaluates the accent error of the assessment object based on the edit distance.

Images