Voice synchronization method of education equipment

Abstract

The invention discloses a voice synchronization method for education equipment, and belongs to the technical field of voice processing, and the method comprises the following steps: S1, collecting user voice through a microphone of early education equipment, and uniformly dividing the user voice into a plurality of frames of voice segments; s2, constructing a voice fluctuation matrix for the user voice according to the plurality of frames of voice segments; and S3, noise of the user voice is eliminated according to the voice fluctuation matrix, and the user voice after noise elimination is synchronously uploaded to a user terminal. The voice synchronization method can improve the voice quality synchronized to the user mobile terminal, simplify the voice de-noising process, and ensure that the synchronized voice is clear.

Description

Technical Field

[0001] The invention belongs to the technical field of voice processing, and in particular relates to a voice synchronization method for educational equipment. Background Art

[0002] As an early childhood education device, the point reading machine is a new generation of electronic voice education products. By clicking on the teaching materials placed on the point reading machine, the text content corresponding to the corresponding position can be played in the form of voice. It is an important auxiliary tool for students to learn, and students can also follow the reading. The voice of the students' follow-up reading needs to be synchronized to the user's mobile terminal (such as a mobile phone or iPad, etc.) in real time for users to view. However, the voice synchronization method of the existing point reading machine cannot clearly clear the noise in the students' follow-up reading voice, which affects the quality of the students' follow-up reading voice. Summary of the invention

[0003] In order to solve the above problems, the present invention proposes a voice synchronization method for educational equipment.

[0004] The technical solution of the present invention is: a voice synchronization method for educational equipment comprises the following steps:

[0005] S1. Use the microphone of the early childhood education device to collect the user's voice and evenly divide the user's voice into several frames of voice segments;

[0006] S2. constructing a voice fluctuation matrix for the user's voice according to a number of voice frames;

[0007] S3. According to the voice fluctuation matrix, the noise of the user's voice is eliminated, and the user's voice after the noise is eliminated is synchronously uploaded to the user terminal.

[0008] Furthermore, S2 includes the following sub-steps:

[0009] S21, in Frame speech segment to A frame of speech segment is randomly selected from the frame speech segment as the standard frame speech segment; wherein L represents the total number of frames of the speech segment, Indicates rounding up operation;

[0010] S23, performing Fourier transform on the standard frame speech segment, the first frame speech segment and the last frame speech segment to obtain the spectrum energy of the standard frame speech segment, the spectrum energy of the first frame speech segment and the spectrum energy of the last frame speech segment respectively;

[0011] S24, calculating the fluctuation coefficient between the first frame speech segment and the standard frame speech segment according to the spectrum energy of the first frame speech segment and the spectrum energy of the standard frame speech segment;

[0012] S25, calculating the fluctuation coefficient between the standard frame speech segment and the last frame speech segment according to the spectrum energy of the standard frame speech segment and the spectrum energy of the last frame speech segment;

[0013] S26. Construct a speech fluctuation matrix according to the fluctuation coefficient between the first frame speech segment and the standard frame speech segment and the fluctuation coefficient between the standard frame speech segment and the last frame speech segment.

[0014] The beneficial effect of the above further scheme is: in the present invention, a frame of speech segment is randomly intercepted from the middle segment of the user's speech, and based on the spectral energy of the frame of speech segment, the fluctuation coefficient between the frame of speech segment and the first frame of speech segment and the last frame of speech segment of the user's speech is calculated, and the final speech fluctuation matrix is ​​determined by the fluctuation coefficient. The speech fluctuation matrix generated in this way can describe the fluctuation of the entire user's speech, taking into account the spectral energy changes of the first and last frames and the middle frames.

[0015] Furthermore, in S24, the calculation formula of the fluctuation coefficient b1 between the first frame speech segment and the standard frame speech segment is:

[0016] ; Where S0 represents the spectral energy of the first frame of speech segment, S d Represents the spectral energy of the standard frame speech segment, S d-1 represents the spectral energy of the previous frame of the standard frame speech segment, S d+1 It represents the spectrum energy of the next frame of speech segment after the standard frame of speech segment, and e represents the exponent.

[0017] Further, in S25, the calculation formula of the fluctuation coefficient b2 between the standard frame speech segment and the last frame speech segment is:

[0018] ; Where S1 represents the spectrum energy of the last frame of speech segment, S d Represents the spectral energy of the standard frame speech segment, S d-1 represents the spectral energy of the previous frame of the standard frame speech segment, S d+1 It represents the spectrum energy of the next frame of speech segment after the standard frame of speech segment, and e represents the exponent.

[0019] Furthermore, in S26, the calculation formula of the speech fluctuation matrix B is:

[0020] ; In the formula, b1 represents the fluctuation coefficient between the first frame speech segment and the standard frame speech segment, and b2 represents the fluctuation coefficient between the standard frame speech segment and the last frame speech segment.

[0021] Furthermore, S3 includes the following sub-steps:

[0022] S31, extracting the short-time energy of each frame of speech segment, and calculating the adjacent variation coefficient of each frame of speech segment;

[0023] S32, determining the noise fluctuation interval according to the voice fluctuation matrix and the adjacent variation coefficients of all frame voice segments;

[0024] S33, processing each frame of voice clips whose short-term energy belongs to the noise fluctuation range to remove the noise of the user's voice;

[0025] S34, synchronously uploading the user voice after noise removal to the user terminal.

[0026] The beneficial effect of the above further scheme is: in the present invention, the adjacent change coefficient of each frame of speech segment is determined by the short-time energy gap between each frame of speech segment and the adjacent frame of speech segment, so as to construct the noise fluctuation interval to screen the frame segments that need to be noise processed, which can simplify the speech processing process and improve processing efficiency.

[0027] Further, in S31, the adjacent change coefficient C of the lth frame speech segment l The calculation formula is:

[0028] ; In the formula, s l represents the short-time energy of the l-th frame speech segment, s l-1 represents the short-time energy of the l-1th frame of speech segment, s l+1 represents the short-time energy of the l+1th frame of speech segment, max(·) represents the maximum value operation, and c represents a constant.

[0029] Further, in S32, the expression of the noise fluctuation interval is [Q1, Q2]; wherein the calculation formula of the left endpoint Q1 of the noise fluctuation interval is:

[0030] ; In the formula, λ b represents the eigenvalue of the speech fluctuation matrix, μ represents the mean of all adjacent variation coefficients, and B represents the speech fluctuation matrix;

[0031] The calculation formula for the right endpoint Q2 of the noise fluctuation range is:

[0032] .

[0033] Furthermore, in S33, the Wiener filtering algorithm is used to process each frame of speech segments whose short-term energy belongs to the noise fluctuation interval.

[0034] The beneficial effects of the present invention are as follows: the voice synchronization method of the educational device segments the user voice collected by the early childhood education device (i.e., the reading machine), generates a voice fluctuation matrix containing the features of the ending frame and the intermediate frame voice segments, determines the noise fluctuation range through the voice fluctuation matrix, screens the voice segments that need to be de-noised, and synchronously uploads them to the user terminal (such as a mobile phone and iPad, etc.); the voice synchronization method can improve the voice quality synchronized to the user's mobile terminal, simplify the voice denoising process, and ensure the clarity of the synchronized voice. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flow chart of a method for speech synchronization for an educational device. DETAILED DESCRIPTION

[0036] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0037] like Figure 1 As shown, the present invention provides a voice synchronization method for educational equipment, comprising the following steps:

[0038] S1. Use the microphone of the early childhood education device to collect the user's voice and evenly divide the user's voice into several frames of voice segments;

[0039] S2. constructing a voice fluctuation matrix for the user's voice according to a number of voice frames;

[0040] S3. According to the voice fluctuation matrix, the noise of the user's voice is eliminated, and the user's voice after the noise is eliminated is synchronously uploaded to the user terminal.

[0041] In this embodiment of the present invention, S2 includes the following sub-steps:

[0042] S21, in Frame speech segment to A frame of speech segment is randomly selected from the frame speech segment as the standard frame speech segment; wherein L represents the total number of frames of the speech segment, Indicates rounding up operation;

[0043] S23, performing Fourier transform on the standard frame speech segment, the first frame speech segment and the last frame speech segment to obtain the spectrum energy of the standard frame speech segment, the spectrum energy of the first frame speech segment and the spectrum energy of the last frame speech segment respectively;

[0044] S24, calculating the fluctuation coefficient between the first frame speech segment and the standard frame speech segment according to the spectrum energy of the first frame speech segment and the spectrum energy of the standard frame speech segment;

[0045] S25, calculating the fluctuation coefficient between the standard frame speech segment and the last frame speech segment according to the spectrum energy of the standard frame speech segment and the spectrum energy of the last frame speech segment;

[0046] S26. Construct a speech fluctuation matrix according to the fluctuation coefficient between the first frame speech segment and the standard frame speech segment and the fluctuation coefficient between the standard frame speech segment and the last frame speech segment.

[0047] In the present invention, a frame of speech segment is randomly intercepted from the middle segment of the user's speech, and according to the spectral energy of the frame of speech segment, the fluctuation coefficient between the frame of speech segment and the first frame of speech segment and the last frame of speech segment of the user's speech is calculated, and the final speech fluctuation matrix is ​​determined by the fluctuation coefficient. The speech fluctuation matrix generated in this way can describe the fluctuation of the entire user's speech, taking into account the spectral energy changes of the first and last frames and the middle frames.

[0048] In the embodiment of the present invention, in S24, the calculation formula of the fluctuation coefficient b1 between the first frame speech segment and the standard frame speech segment is:

[0049] ; Where S0 represents the spectral energy of the first frame of speech segment, S d Represents the spectral energy of the standard frame speech segment, S d-1 represents the spectral energy of the previous frame of the standard frame speech segment, S d+1 It represents the spectrum energy of the next frame of speech segment after the standard frame of speech segment, and e represents the exponent.

[0050] In the embodiment of the present invention, in S25, the calculation formula of the fluctuation coefficient b2 between the standard frame speech segment and the last frame speech segment is:

[0051] ; Where S1 represents the spectrum energy of the last frame of speech segment, S d Represents the spectral energy of the standard frame speech segment, S d-1 represents the spectral energy of the previous frame of the standard frame speech segment, S d+1 It represents the spectrum energy of the next frame of speech segment after the standard frame of speech segment, and e represents the exponent.

[0052] In the embodiment of the present invention, in S26, the calculation formula of the voice fluctuation matrix B is: ; In the formula, b1 represents the fluctuation coefficient between the first frame speech segment and the standard frame speech segment, and b2 represents the fluctuation coefficient between the standard frame speech segment and the last frame speech segment.

[0053] In this embodiment of the present invention, S3 includes the following sub-steps:

[0054] S31, extracting the short-time energy of each frame of speech segment, and calculating the adjacent variation coefficient of each frame of speech segment;

[0055] S32, determining the noise fluctuation interval according to the voice fluctuation matrix and the adjacent variation coefficients of all frame voice segments;

[0056] S33, processing each frame of voice clips whose short-term energy belongs to the noise fluctuation range to remove the noise of the user's voice;

[0057] S34, synchronously uploading the user voice after noise removal to the user terminal.

[0058] In the present invention, the adjacent variation coefficient of each frame of speech segment is determined based on the short-time energy gap between each frame of speech segment and the adjacent frame of speech segment, so as to construct the noise fluctuation interval to screen the frame segments that need to be processed by noise, which can simplify the speech processing process and improve the processing efficiency.

[0059] In the embodiment of the present invention, in S31, the adjacent change coefficient C of the lth frame speech segment l The calculation formula is:

[0060] ; In the formula, s l represents the short-time energy of the l-th frame speech segment, s l-1 represents the short-time energy of the l-1th frame of speech segment, s l+1 represents the short-time energy of the l+1th frame of speech segment, max(·) represents the maximum value operation, and c represents a constant.

[0061] In the embodiment of the present invention, in S32, the expression of the noise fluctuation interval is [Q1, Q2]; wherein the calculation formula of the left endpoint Q1 of the noise fluctuation interval is:

[0062] ; In the formula, λ b represents the eigenvalue of the speech fluctuation matrix, μ represents the mean of all adjacent variation coefficients, and B represents the speech fluctuation matrix;

[0063] The calculation formula for the right endpoint Q2 of the noise fluctuation range is:

[0064] .

[0065] In the embodiment of the present invention, in S33, the Wiener filtering algorithm is used to process each frame of speech segments whose short-term energy belongs to the noise fluctuation interval.

[0066] Wiener filtering is a classic speech uplink denoising algorithm. It is based on the minimum mean square error criterion and estimates the power spectral density of speech and noise signals and adjusts the filter gain according to the ratio of the signal-to-noise ratio to minimize the mean square error between the estimated speech signal and the observed speech signal.

[0067] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A voice synchronization method for educational equipment, characterized in that: The following steps are involved: S1. Use the microphone of the early childhood education device to collect the user's voice and evenly divide the user's voice into several frames of voice segments; S2. constructing a voice fluctuation matrix for the user's voice according to a number of voice frames; S3. According to the voice fluctuation matrix, the noise of the user's voice is eliminated, and the user's voice after the noise is eliminated is synchronously uploaded to the user terminal.

2. The method for voice synchronization of educational equipment according to claim 1, characterized in that: The S2 comprises the following sub-steps: S21, in Frame speech segment to A frame of speech segment is randomly selected from the frame speech segment as the standard frame speech segment; wherein L represents the total number of frames of the speech segment, Indicates rounding up operation; S23, performing Fourier transform on the standard frame speech segment, the first frame speech segment and the last frame speech segment to obtain the spectrum energy of the standard frame speech segment, the spectrum energy of the first frame speech segment and the spectrum energy of the last frame speech segment respectively; S24, calculating the fluctuation coefficient between the first frame speech segment and the standard frame speech segment according to the spectrum energy of the first frame speech segment and the spectrum energy of the standard frame speech segment; S25, calculating the fluctuation coefficient between the standard frame speech segment and the last frame speech segment according to the spectrum energy of the standard frame speech segment and the spectrum energy of the last frame speech segment; S26. Construct a speech fluctuation matrix according to the fluctuation coefficient between the first frame speech segment and the standard frame speech segment and the fluctuation coefficient between the standard frame speech segment and the last frame speech segment.

3. The method for voice synchronization of educational equipment according to claim 2, characterized in that: In S24, the calculation formula of the fluctuation coefficient b1 between the first frame speech segment and the standard frame speech segment is: ; Where S0 represents the spectral energy of the first frame of speech segment, S d Represents the spectral energy of the standard frame speech segment, S d-1 represents the spectral energy of the previous frame of the standard frame speech segment, S d+1 It represents the spectrum energy of the next frame of speech segment after the standard frame of speech segment, and e represents the exponent.

4. The method for voice synchronization of educational equipment according to claim 2, characterized in that: In S25, the calculation formula of the fluctuation coefficient b2 between the standard frame speech segment and the last frame speech segment is: ; Where S1 represents the spectrum energy of the last frame of speech segment, S d Represents the spectral energy of the standard frame speech segment, S d-1 represents the spectral energy of the previous frame of the standard frame speech segment, S d+1 It represents the spectrum energy of the next frame of speech segment after the standard frame of speech segment, and e represents the exponent.

5. The method for voice synchronization of educational equipment according to claim 2, characterized in that: In S26, the calculation formula of the voice fluctuation matrix B is: ; In the formula, b1 represents the fluctuation coefficient between the first frame speech segment and the standard frame speech segment, and b2 represents the fluctuation coefficient between the standard frame speech segment and the last frame speech segment.

6. The method for voice synchronization of educational equipment according to claim 1, characterized in that: The S3 comprises the following sub-steps: S31, extracting the short-time energy of each frame of speech segment, and calculating the adjacent variation coefficient of each frame of speech segment; S32, determining the noise fluctuation interval according to the voice fluctuation matrix and the adjacent variation coefficients of all frame voice segments; S33, processing each frame of voice clips whose short-term energy belongs to the noise fluctuation range to remove the noise of the user's voice; S34, synchronously uploading the user voice after noise removal to the user terminal.

7. The method for voice synchronization of educational equipment according to claim 6, characterized in that: In the step S31, the adjacent change coefficient C of the lth frame speech segment l The calculation formula is: ; In the formula, s l represents the short-time energy of the l-th frame of speech segment, s l-1 represents the short-time energy of the l-1th frame of speech segment, s l+1 represents the short-time energy of the l+1th frame of speech segment, max(·) represents the maximum value operation, and c represents a constant.

8. The method for voice synchronization of educational equipment according to claim 6, characterized in that: In S32, the expression of the noise fluctuation interval is [Q1, Q2]; wherein the calculation formula of the left endpoint Q1 of the noise fluctuation interval is: ; In the formula, λ b represents the eigenvalue of the speech fluctuation matrix, μ represents the mean of all adjacent variation coefficients, and B represents the speech fluctuation matrix; The calculation formula for the right endpoint Q2 of the noise fluctuation range is: 。 9. The method for voice synchronization of educational equipment according to claim 6, characterized in that: In S33, the Wiener filtering algorithm is used to process each frame of speech segments whose short-term energy belongs to the noise fluctuation interval.

Images