Chinese accent error assessment method and system based on error dictionary and sequence alignment

Abstract

The present invention discloses a Chinese accent error assessment method and system based on an error dictionary and sequence alignment, and relates to the technical field of text analysis. The method comprises the following steps: collecting a speech signal of an assessment object reading a standard phoneme text, identifying and generating a phoneme text to be assessed; comparing the phoneme text to be assessed and the standard phoneme text, and constructing an accent error dictionary; calculating the edit distance between the phoneme text to be assessed and the standard phoneme text based on the accent error dictionary; and assessing the accent error of the assessment object based on the edit distance. The present invention constructs a detailed accent error dictionary, and on this basis, uses the Levenshtein sequence alignment algorithm to calculate the edit distance between the text to be assessed and the standard text, and then calculates the dictionary error edit distance and other error edit distances based on the dictionary; a comprehensive error quantification score is obtained by weighting the two distances, and the score can more accurately reflect the degree of the learner's accent error.

Description

Technical Field

[0001] The present invention relates to the technical field of text analysis, and in particular to a Chinese accent error assessment method and system based on an error dictionary and sequence alignment. Background Art

[0002] In the process of learning Chinese, non-native learners often make errors in intonation and pronunciation due to factors such as the influence of their mother tongue, age, and teaching methods. Analyzing and evaluating the accent errors of non-native learners can help us gain a deeper understanding of their mastery of the target language. By analyzing specific phonetic errors, such as initial consonants, finals, and tone errors, learners can practice and overcome these difficulties in a targeted manner. In short, error analysis helps improve the quality of international Chinese teaching, promote the understanding of the second language acquisition mechanism, gain a deeper understanding of students' needs, and provide targeted improvement strategies for teaching.

[0003] Accent error assessment research currently faces many challenges. First, accent error assessment research often relies on limited data sets, which may lead to insufficient generalization of the assessment results. Second, current research focuses on a single accent or dialect, and lacks research on the comprehensive assessment of multiple accents. In addition, there is no unified assessment standard for accent errors, and different studies may use different assessment methods, which makes it difficult to compare and synthesize research. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of lack of evaluation criteria for accent errors in the prior art.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a Chinese accent error assessment method based on an error dictionary and sequence alignment, comprising the following steps:

[0006] Collect the speech signal of the evaluation object reading the standard phoneme text, and recognize and generate the phoneme text to be evaluated;

[0007] Compare the phoneme text to be evaluated with the standard phoneme text and construct an accent error dictionary;

[0008] Calculate the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary;

[0009] The accent error of the evaluation object is evaluated based on the edit distance.

[0010] Preferably, the construction of the accent error dictionary comprises the following steps:

[0011] Compare the phoneme text to be evaluated with the standard phoneme text, count the incorrectly mapped syllable pairs, and filter out the syllable pairs with a frequency higher than the preset threshold Error mapping, get the high-frequency error mapping set ,in, Indicates the wrong speech recognition phoneme, Indicates standard phonemes;

[0012] Mapping high-frequency errors to sets and the Global Accent Error Dataset Perform integration to obtain an accent error dictionary , the Global Accent Error Dataset Contains syllable substitution patterns of pronunciation errors in different countries.

[0013] Preferably, the method of calculating the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary adopts the Levenshtein sequence alignment algorithm, and includes the following steps:

[0014] The standard phoneme text is used as the A sequence, and the phoneme text to be evaluated is used as the B sequence;

[0015] Initialize the matrix and set the size of the matrix to (m+1)×(n+1), where m is the length of sequence A and n is the length of sequence B; the first row and the first column of the matrix are initialized to the sequence from 0 to n and the sequence from 0 to m respectively;

[0016] Fill the matrix, starting from i=1, j=1, and gradually fill each position (i, j) in the matrix; i∈[0, m] represents the row index, j∈[0, n] represents the column index; the value dp[i][j] filled to the position (i, j) represents the edit distance, which is determined by the comparison result of the i-th element of sequence A and the j-th element of sequence B;

[0017] After completing the filling of the matrix, take (m, n) as the backtracking point, calculate the difference between the value of the backtracking point and the minimum value in the adjacent position of the backtracking point as the distance, and use the position of the minimum value as the new backtracking point to continue calculating the distance until the position (0, 0), and then connect the distances of all backtracking points as a sequence representing the minimum editing operation path; for the backtracking point (i, j), its adjacent positions include (i-1, j), (i, j-1), (i-1, j-1);

[0018] The edit distance is calculated based on the accent error dictionary in the sequence representing the minimum editing operation path.

[0019] Preferably, the value filled into the position (i, j) is determined according to the comparison result of the i-th element of the A sequence and the j-th element of the B sequence, specifically:

[0020] if equal ,but , indicating that no operation is required;

[0021] if Not equal to ,but , indicating that editing is required; among them, Represents an insert operation. Indicates a delete operation. Indicates a replacement operation; , and Both represent operation weights, and , .

[0022] Preferably, the step of calculating the edit distance in a sequence representing a minimum edit operation path based on an accent error dictionary comprises:

[0023] Classifying the edit distances in the sequence representing the minimum edit operation path, if the edit operation corresponding to the edit distance is a replacement operation and the replacement operation conforms to the error mapping of the accent error dictionary D, then classifying the edit distance as a dictionary error edit distance; otherwise, classifying the edit distance as an error edit distance;

[0024] Classify and add the edit distances in the sequence representing the minimum edit operation path, and add all dictionary error edit distances to get the total dictionary error edit distance. , add up all the error edit distances to get the total error edit distance .

[0025] Preferably, the method of evaluating the accent error of the evaluation object based on the edit distance is specifically as follows: assigning different weights to different types of edit distances, and adding the weights of the edit distances to obtain an accent error score, wherein a higher score indicates a more serious pronunciation error of the evaluation object; the accent error score It is expressed as:

[0026] ;

[0027] in, for The weight of for The weight of .

[0028] The present invention also provides a Chinese accent error assessment system based on an error dictionary and sequence alignment, comprising:

[0029] The voice collection module collects the voice signal of the evaluation object reading the standard phoneme text, performs voice recognition to generate the phoneme text to be evaluated;

[0030] The dictionary building module compares the phoneme text to be evaluated with the standard phoneme text to build an accent error dictionary;

[0031] A distance calculation module calculates the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary;

[0032] The accent scoring module evaluates the accent error of the assessment object based on the edit distance.

[0033] The present invention has the following beneficial effects:

[0034] (1) The present invention constructs a detailed accent error dictionary. On this basis, the Levenshtein sequence alignment algorithm is used to calculate the edit distance between the phoneme text obtained by speech recognition and the standard phoneme text, and the algorithm's backtracking mechanism is used to count the edit distance of the errors in the dictionary and other incorrect edit distances; by weighting these two distances, a comprehensive error quantification score is obtained, which can more accurately reflect the degree of the learner's accent error; this method not only focuses on the applicability of oral communication for non-native Chinese learners, but also provides targeted improvement suggestions for teaching and learning through precise quantitative evaluation. Through the application of this method, the quality and effect of Chinese teaching can be significantly improved, and the learners' oral ability can be further improved.

[0035] (2) The innovation of the present invention lies in its in-depth analysis and quantitative evaluation of the error recognition process, which provides a new perspective and tool for the teaching of Chinese as a second language. Through the systematic study and quantification of errors, different editing weights are assigned to different editing operations (for phonemes, if replacement is required, it means that the difference between the two phoneme sequences is large, and the pronunciation error reflected is greater. Giving a larger editing weight to replacement can better reflect the degree of error), which helps teachers to more effectively identify and correct learners' oral errors, and also provides learners with clear learning goals and feedback, thereby promoting two-way interaction and optimization of teaching and learning.

[0036] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A method step diagram of an embodiment of the present invention;

[0038] Figure 2 4 is a system architecture diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0039] See also Figure 1 FIG. 1 is a diagram showing steps of a method according to an embodiment of the present invention, comprising the following steps:

[0040] S101, collecting a speech signal of the evaluation object reading a standard phoneme text, and performing speech recognition to generate a phoneme text to be evaluated;

[0041] S102, comparing the phoneme text to be evaluated with the standard phoneme text, and constructing an accent error dictionary;

[0042] S103, calculating the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary;

[0043] S104, evaluating the accent error of the evaluation object based on the edit distance.

[0044] Specifically, the step S102 includes the following steps:

[0045] S1021, compare the phoneme text to be evaluated with the standard phoneme text, count the incorrectly mapped syllable pairs, and filter out the syllable pairs with a frequency higher than a preset threshold Error mappings such as , get the high-frequency error mapping set ,in, Indicates the wrong speech recognition phoneme, represents the standard phoneme; the data set used in this embodiment is , N=22753 is the size of the dataset. Examples of the statistics of some syllable substitutions in the text are shown in Table 1.

[0046] Table 1 Example of syllable substitution statistics:

[0047]

[0048] S1022, mapping high frequency errors to sets and the Global Accent Error Dataset Perform integration to obtain an accent error dictionary , the Global Accent Error Dataset Contains syllable substitution patterns of pronunciation errors in different countries; among them, The typical pronunciation errors of learners from different countries provided by the "Global Accent Chinese" project (https: / / www.globalaccentchinese.com / ) are used to verify the syllable substitution patterns obtained from actual speech data. The dictionary contains syllable substitution patterns of pronunciation errors in different countries, as shown in Table 2, which are examples of pronunciation errors in Thai and Vietnamese accents.

[0049] Table 2 Examples of pronunciation errors in Thai and Vietnamese accents:

[0050]

[0051] Non-native Chinese learners in Thailand often use the following substitution rules when pronouncing Chinese: h k;zh z;sh c; in terms of finals: a o;ia ie;ui ao; in tone: 5 . Non-native Chinese learners in Vietnam usually use the following substitution rules when pronouncing initial consonants: p d, b p,d t,c ch,zh z, q j,ch q; in terms of finals: e o,ou uo; in terms of tone: 4 The phonemes on the left side of the arrow represent the phonemes that are often mispronounced, while the phonemes on the right side of the arrow are the corrected phonemes.

[0052] Specifically, the S103 includes the following steps:

[0053] S1031, using the standard phoneme text as the A sequence and the phoneme text to be evaluated as the B sequence; this embodiment takes A = "shi4 jie4 de5", B = "shi4 qie4 de1" as an example for explanation;

[0054] S1032, initialize a two-dimensional matrix dp, set the size of the matrix to (m+1)×(n+1), where m is the length of sequence A and n is the length of sequence B; in this embodiment, the size of the matrix is ​​4×4, and the first row and the first column of the matrix are initialized to sequences from 0 to 3 respectively;

[0055] S1033, fill the matrix, starting from i=1, j=1, and gradually fill each position (i, j) in the matrix; the value dp[i][j] filled to the position (i, j) represents the edit distance, which is determined by the comparison result of the i-th element of sequence A and the j-th element of sequence B; i∈[0, m] represents the row index, j∈[0, n] represents the column index, but note that the filling of this step starts from i=1, j=1;

[0056] S1034, after completing the filling of the matrix, take (m, n) as the backtracking point, calculate the difference between the value of the backtracking point and the minimum value in the adjacent position of the backtracking point as the distance, and take the position of the minimum value as the new backtracking point to continue calculating the distance until the position (0, 0), and then connect the distances of all the backtracking points as a sequence representing the minimum editing operation path; for the backtracking point (i, j), its adjacent positions include (i-1, j), (i, j-1), (i-1, j-1);

[0057] S1035: Calculate edit distance according to the minimum edit operation path.

[0058] Specifically, the S1033 is as follows: if equal ,but , indicating that no action is required; if Not equal to ,but , indicating that an edit operation is required and the edit operation with the smallest edit distance is selected; there are three types of edit operations: Represents an insert operation. Indicates a delete operation. Indicates a replacement operation; , and Both represent operation weights, and , .For example, ,express The "shi4" and The "shi4" in the sequence is equal and no editing operation is required; , ,but , which means that the edit distance of replacing "shi4" with "qie4" is 1 (the edit distance of the insertion operation is 3, the edit distance of the replacement operation is 1, and the edit distance of the deletion operation is 2, so the minimum value 1 is selected as The filled matrix is ​​shown in Table 3.

[0059] Table 3 Edit distance table:

[0060]

[0061] Specifically, in S1034, backtracking starts from the lower right corner as the first backtracking point Matrix, find the sequence representing the minimum editing operation path during the backtracking process, that is, find the values ​​of the adjacent positions (top, left, top left corner) of each backtracking point in the matrix, calculate the difference between the value of the backtracking point and the minimum value among them as the distance, and use the position of the minimum value as the new backtracking point to continue calculating the distance; connect the distance of each backtracking point as the sequence of the latest operation path. In this embodiment, the backtracking point dp[3][3]=4, and the minimum value in its adjacent position is 2 (i.e., dp[2][2]). The difference between the backtracking points dp[3][3] and dp[2][2] is 2 as the distance; dp[2][2] is taken as the new backtracking point, and the minimum value in its adjacent position is 0 (i.e., dp[1][1]). The difference between the backtracking points dp[2][2] and dp[1][1] is 2 as the distance; dp[1][1] is taken as the new backtracking point, and the minimum value in its adjacent position is 0 (i.e., dp[0][0]). The difference between the backtracking points dp[1][1] and dp[0][0] is 0 as the distance; therefore, the sequence 2, 2, 0 is finally obtained, and the alignment result table based on the sequence is shown in Table 4.

[0062] Table 4 Alignment results table:

[0063]

[0064] Specifically, the dictionary error edit distance is calculated in S1035 and error edit distance Among them, the dictionary error edit distance is 2, which means "shi4" is replaced by "qie4"; Wrong edit distance is 2, which means "de1" is replaced by "de5".

[0065] Specifically, S104 is as follows: assigning different weights to different types of edit distances, adding the weights of the edit distances, and obtaining a result as an accent error score. A higher score indicates a more serious pronunciation error of the non-native speaker. for The weight of =0.7 The weight of; According to Table 4, the dictionary error edit distance is obtained 2, wrong edit distance If it is 2, the weighted quantitative evaluation score of the accent error is The calculation is as follows:

[0066]

[0067] See also Figure 2 FIG. 1 is a system structure diagram of an embodiment of the present invention, including:

[0068] The speech collection module 201 collects the speech signal of the evaluation object reading the standard phoneme text, performs speech recognition to generate the phoneme text to be evaluated;

[0069] A dictionary building module 202 compares the phoneme text to be evaluated with the standard phoneme text to build an accent error dictionary;

[0070] A distance calculation module 203 calculates the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary;

[0071] The accent scoring module 204 evaluates the accent error of the assessment object based on the edit distance.

[0072] The present invention pays full attention to the communicative applicability of spoken language of non-native Chinese learners, improves the accuracy of accent error recognition, and provides specific improvement suggestions for teaching and learning, thereby improving the quality and effect of Chinese teaching.

[0073] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A Chinese accent error assessment method based on error dictionary and sequence alignment, characterized in that: The steps include: Collect the speech signal of the evaluation object reading the standard phoneme text, and recognize and generate the phoneme text to be evaluated; Compare the phoneme text to be evaluated with the standard phoneme text and construct an accent error dictionary; Calculate the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary; Evaluate the accent bias of the subject based on the edit distance; The construction of the accent error dictionary comprises the following steps: By comparing the phoneme text to be evaluated with the standard phoneme text, the syllable pairs with incorrect mappings are counted, and the error mappings with a frequency higher than the preset threshold θ are screened to obtain a high-frequency error mapping set D s ={phm stu →phm std }, where phm stu Indicates incorrect speech recognition phoneme, phm std Indicates standard phonemes; Map high-frequency errors to the set D s and the global accent error dataset D e The global accent error dataset D is integrated to obtain the accent error dictionary D. e Contains syllable substitution patterns of pronunciation errors in different countries.

2. The Chinese accent error assessment method based on error dictionary and sequence alignment according to claim 1, characterized in that: The method of calculating the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary adopts the Levenshtein sequence alignment algorithm, including the following steps: The standard phoneme text is used as the A sequence, and the phoneme text to be evaluated is used as the B sequence; Initialize the matrix and set the size of the matrix to (m+1)×(n+1), where m is the length of sequence A and n is the length of sequence B; the first row and the first column of the matrix are initialized to the sequence from 0 to n and the sequence from 0 to m respectively; Fill the matrix, starting from i=1, j=1, and gradually fill each position (i, j) in the matrix; i∈[0, m] represents the row index, j∈[0, n] represents the column index; the value dp[i][j] filled to the position (i, j) represents the edit distance, which is determined by the comparison result of the i-th element of sequence A and the j-th element of sequence B; After completing the filling of the matrix, take (m, n) as the backtracking point, calculate the difference between the value of the backtracking point and the minimum value in the adjacent position of the backtracking point as the distance, and use the position of the minimum value as the new backtracking point to continue calculating the distance until the position (0, 0), and then connect the distances of all backtracking points as a sequence representing the minimum editing operation path; for the backtracking point (i, j), its adjacent positions include (i-1, j), (i, j-1), (i-1, j-1); The edit distance is calculated based on the accent error dictionary in the sequence representing the minimum editing operation path.

3. The Chinese accent error assessment method based on error dictionary and sequence alignment according to claim 2, characterized in that: The value filled to the position (i, j) is determined according to the comparison result of the i-th element of the A sequence and the j-th element of the B sequence, specifically: If A[i] is equal to B[j], then dp[i][j] = dp[i-1][j-1], which means no operation is required; If A[i] is not equal to B[j], then dp[i][j]=min(dp[i-1][j]+β1,dp[i][j-1]+β2,dp[i-1][j-1]+β3), indicating that an editing operation is required; among them, dp[i][j]=dp[i-1][j] indicates an insertion operation, dp[i][j]=dp[i][j-1] indicates a deletion operation, and dp[i][j]=dp[i-1][j-1] indicates a replacement operation; β1, β2 and β3 all represent operation weights, and β1<β3, β2<β3.

4. The Chinese accent error assessment method based on error dictionary and sequence alignment according to claim 3 is characterized in that: The method of calculating the edit distance in a sequence representing a minimum edit operation path based on an accent error dictionary includes: Classifying the edit distances in the sequence representing the minimum edit operation path, if the edit operation corresponding to the edit distance is a replacement operation and the replacement operation conforms to the error mapping of the accent error dictionary D, then classifying the edit distance as a dictionary error edit distance; otherwise, classifying the edit distance as an error edit distance; The edit distances in the sequence representing the minimum edit operation path are classified and added, and all dictionary error edit distances are added to obtain the total dictionary error edit distance dis dict , add up all the error edit distances to get the total error edit distance dis err .

5. The Chinese accent error assessment method based on error dictionary and sequence alignment according to claim 3, characterized in that: The method of evaluating the accent bias of the evaluation object based on the edit distance is specifically as follows: assigning different weights to different types of edit distances, and adding the weights of the edit distances to obtain an accent bias score. The higher the score, the more serious the pronunciation bias of the evaluation object. The accent bias score ScoreBias is expressed as: ScoreBias=dis dict ×ɑ1+dis err ×ɑ2; Among them, ɑ1 is dis dict The weight of ɑ2 is dis err The weight of .

6. A Chinese accent error assessment system based on error dictionary and sequence alignment, characterized in that: include: The voice collection module collects the voice signal of the evaluation object reading the standard phoneme text, performs voice recognition to generate the phoneme text to be evaluated; The dictionary building module compares the phoneme text to be evaluated with the standard phoneme text to build an accent error dictionary; A distance calculation module calculates the edit distance between the phoneme text to be evaluated and the standard phoneme text based on the accent error dictionary; The accent scoring module evaluates the accent error of the subject based on the edit distance; The construction of the accent error dictionary comprises the following steps: By comparing the phoneme text to be evaluated with the standard phoneme text, the syllable pairs with incorrect mappings are counted, and the error mappings with a frequency higher than the preset threshold θ are screened to obtain a high-frequency error mapping set D s = {Phm stu →Phm std }, where Phm stu Indicates the wrong speech recognition phoneme, Phm std Indicates standard phonemes; Map high-frequency errors to the set D s and the global accent error dataset D e The global accent error dataset D is integrated to obtain the accent error dictionary D. e Contains syllable substitution patterns of pronunciation errors in different countries.

Images