A method for implementing multilingual simultaneous interpretation agent assistance in a noisy environment

Abstract

The invention belongs to the field of speech recognition, and specifically discloses a method for realizing multi-language simultaneous interpretation seat assistance adapting to a noisy environment, comprising the following steps: S1 dividing a sampling data frame: determining a time interval value, and dividing the continuous sampling data of a speech signal into a plurality of data frames; S2 calculating sound energy: processing the sampling data in each data frame, and calculating its sound energy, that is, the mean of the absolute value of the sampling data; S3 calculating a fast and slow moving average value: processing the sound energy in each data frame, and calculating the fast and slow moving average values ​​respectively; S4 judging the start and end of speech: judging the start and end of speech according to the change of the fast and slow moving average values. The invention improves the accuracy and adaptability of the start and end of sound by introducing a method of fast / slow moving to average value difference, reduces the risk of false detection and missed detection, and thus improves the performance of a speech recognition system.

Description

Technical Field

[0001] The invention belongs to the field of speech recognition, and specifically discloses a method for realizing multi-language simultaneous interpretation agent assistance adapting to noisy environments. Technical Background

[0002] Currently, many companies and research institutions are working to improve the performance of speech recognition systems in various environments and reduce the impact of noise on speech recognition. However, these systems may use fixed thresholds or simple moving averages to detect the start and end of a sound. For example, a common detection method continuously calculates the average value (sound energy) of the absolute value of sound samples over a short period of time and compares it with a set threshold to determine the start and end times of the sound. This method has the following disadvantages: it is difficult to select an appropriate threshold that can be used in both quiet and noisy environments. Furthermore, the fixed threshold used in traditional methods cannot effectively detect the start and end of a sound in different noise levels, which can easily lead to false or missed detections. Moreover, traditional methods only consider a single moving average, making it difficult to accurately detect sounds in dynamic environments. Traditional methods often have low accuracy in noisy environments, making them difficult to adapt to practical application requirements. Summary of the Invention

[0003] To address the above problems, the present invention discloses a method for assisting multilingual simultaneous interpretation agents in noisy environments. The method combines technical means such as a method of fast / slow moving to the average value difference and a strategy of dynamically adjusting the threshold, thereby achieving the purpose of more effectively detecting the start and end positions of sounds in noisy environments.

[0004] The present invention includes the following technical solutions:

[0005] A method for implementing multilingual simultaneous interpretation agent assistance in a noisy environment includes the following steps:

[0006] S1 divides the sampled data into frames: determines a time interval value and divides the continuous sampled data of the speech signal into several data frames;

[0007] S2 calculates sound energy: the sampled data in each data frame is processed and its sound energy is calculated, that is, the mean of the absolute values ​​of the sampled data; the purpose of this step is to obtain the sound intensity information in each data frame as the basis for subsequent calculations;

[0008] S3 calculates fast and slow moving averages: The sound energy in each data frame is processed and fast and slow moving averages are calculated respectively. The fast moving average uses a larger new frame coefficient to more sensitively track changes in sound energy, while the slow moving average uses a smaller new frame coefficient to more smoothly reflect the long-term trend of sound energy changes.

[0009] S4 determines the start and end of speech: the start and end of speech are determined based on the changes in the fast and slow moving averages.

[0010] Furthermore, in the above-mentioned method for assisting multilingual simultaneous interpreting agents in a noisy environment, in step S3, the fast moving average uses a larger new frame coefficient to more sensitively track changes in sound energy, while the slow moving average uses a smaller new frame coefficient to more smoothly reflect the long-term trend of sound energy changes;

[0011] The new frame coefficient is a value between 0 and 100 (exclusive), representing the proportion of the acoustic moving sound energy of the latest frame of speech data in the moving average;

[0012] The calculation formula of the moving average is: Moving average of this frame = new frame coefficient * sound energy of this frame + (100-new frame coefficient) * moving average of the previous frame.

[0013] Furthermore, in the above-mentioned method for assisting multilingual simultaneous interpreting agents in a noisy environment, the new frame coefficient is an adjustable parameter, and the new frame coefficients of the fast and slow moving averages can be respectively set to 30 and 10 based on experience.

[0014] Furthermore, in the above-mentioned method for assisting multilingual simultaneous interpreters in a noisy environment, in step S4, when the fast moving average value changes from less than (slow moving average value + threshold value) to greater than (slow moving average value + threshold value), it is determined that the sound has started; and when the fast moving average value changes from greater than (slow moving average value - threshold value) to less than (slow moving average value - threshold value), it is determined that the sound has ended.

[0015] Furthermore, in the aforementioned method for assisting multilingual simultaneous interpreters in noisy environments, the threshold in step S4 is dynamically adjusted to accommodate voice detection requirements in varying noise environments. In this step, the difference between the fast and slow moving averages is adjusted based on the noise level in the actual scene, thereby adaptively adjusting the threshold and making voice detection more accurate and reliable.

[0016] Preferably, two thresholds are used to determine the start and end of speech: a sound energy variation threshold and a pause interval threshold. The amplitude of sound energy variation is represented by the difference between the fast moving average and the slow moving average. In the non-speaking state, as the voice data stream arrives, if the difference between the fast moving average and the slow moving average of the current data frame is greater than the sound energy variation amplitude threshold, the current time point is marked as the speech start reference point and the state is switched to the speaking state (because the moving average has a lag, the speech data must be captured back a period of time, for example, 200 milliseconds, from the speech start reference point). In the speaking state, if the difference between the fast moving average and the slow moving average of the current data frame is less than the negative value of the sound energy variation amplitude threshold, the current time point is marked as the speech end reference point. Starting from the speech end reference point, if the difference between the fast moving average and the slow moving average of the current data frame is again detected to be greater than the sound energy variation amplitude threshold within the time period specified by the pause interval threshold, the speech is considered incomplete and the previously set speech end reference point is invalidated. Otherwise, the speech is determined to have ended, the previously set speech end reference point is confirmed as the speech end time point, and the state is switched to the non-speaking state.

[0017] On the other hand, the present invention discloses a detection device that can adapt to noisy environments and realize multi-language simultaneous interpretation agent assistance. The firmware part of the detection device contains a software algorithm that realizes the above method.

[0018] Furthermore, the above detection device includes the following specific modules: a data acquisition module, a data processing module, a sound energy calculation module, a moving average calculation module, and a speech start and end judgment module;

[0019] The data acquisition module is responsible for acquiring voice data from the input device;

[0020] The data processing module pre-processes the collected voice data;

[0021] The sound energy calculation module calculates the sound energy of each data frame;

[0022] The moving average calculation module calculates fast and slow moving averages;

[0023] The speech start and end judgment module judges the start and end of speech according to the change of the moving average value.

[0024] On the other hand, the present invention also discloses the application of the above method or detection device in multilingual simultaneous interpretation call center seats.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1) Use the fast / slow moving to average difference method:

[0027] Improvement: Compared with the traditional method of fixed threshold judgment, the present invention adopts the method of fast / slow moving to the average value difference, so that the threshold can be adaptively adjusted according to the degree of environmental noise.

[0028] Technical problem: The fixed threshold in traditional methods makes it difficult to effectively detect the beginning and end of sounds under different noise levels, which can easily lead to false detection or missed detection.

[0029] Advantages: By dynamically adjusting the threshold, the detection of the beginning and end of the sound is more flexible and accurate, and can adapt to the needs of speech recognition in different noise environments.

[0030] 2) Fast and slow moving averages:

[0031] Improvements: The concept of fast and slow moving averages is introduced, and the difference between the two is used to determine the start and end of a sound, which improves the accuracy and stability of detection.

[0032] Technical problem: Traditional methods only consider a single moving average, which makes it difficult to accurately detect sound in a dynamic environment.

[0033] Advantages: By combining fast and slow moving averages, the influence of noise can be effectively suppressed, and the stability and robustness of detection are improved.

[0034] 3) Adaptability and accuracy:

[0035] Improvement: This invention proposes a more adaptable and accurate detection method for speech recognition scenarios in different noise environments.

[0036] Technical issues: Traditional methods often have low accuracy in noisy environments and are difficult to adapt to practical application needs.

[0037] Advantages: By using the method of fast / slow moving to the average value difference, it can more effectively meet the needs of speech recognition in different noise environments, and improve the accuracy and reliability of detection.

[0038] In summary, the present invention effectively solves the technical problems of difficult threshold selection, inaccurate single moving average value, and poor adaptability in traditional methods by introducing a method of fast / slow moving to average value difference, combining fast and slow moving average values, and improving the detection method of adaptability and accuracy. It has higher practical value and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1: In a quiet environment, a multilingual simultaneous interpreter is used to detect a single sentence. For example, the sound energy change threshold is 200 and the pause interval threshold is 1000 milliseconds. Each frame of data contains 30 milliseconds of speech.

[0040] Figure 2 : In a noisy environment, a multilingual simultaneous interpretation assistant detects a sentence being spoken. For example, a sound energy change threshold of 200 milliseconds and a pause interval threshold of 1000 milliseconds are used, and each frame of data contains 30 milliseconds of speech. DETAILED DESCRIPTION

[0041] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0042] Example 1

[0043] Multilingual simultaneous interpretation agent assistance system in a quiet environment (see Figure 1 ), including the following steps:

[0044] Data collection and preprocessing: First, voice data is collected from input devices (such as microphones) and undergoes preprocessing steps such as noise removal and normalization to ensure the accuracy and stability of subsequent processing.

[0045] S1 Data Frame Division: This system uses the SIP protocol for voice communication and adopts the default SIP packet interval parameter of 20 milliseconds. Therefore, data frames are divided into 20-millisecond intervals to perfectly match SIP. Each point on the horizontal axis in the figure represents a data frame, which will serve as the basic unit for subsequent processing.

[0046] S2 Sound Energy Calculation: This process processes the speech signal in each data frame and calculates its sound energy, which is the mean of the absolute values ​​of the sampled data. This is used for subsequent moving average calculations. Each point on the blue curve in the figure represents the sound energy value for that frame.

[0047] S3 Moving Average Calculation: The new frame coefficients for the fast and slow moving averages are set to 30 and 10, respectively. The fast and slow moving averages are calculated for each frame in real time. In the figure, each point on the gray curve represents the fast moving average for that frame, and each point on the orange curve represents the slow moving average for that frame.

[0048] S4 speech start and end judgment: Determine the start and end of speech based on the difference between the fast and slow moving averages and the dynamically adjusted threshold.

[0049] In the figure, the difference between the fast and slow moving averages in frame 21 is 212, exceeding the set threshold of 200. This frame is designated as the speech start reference point. The difference between the fast and slow moving averages in frame 72 is -207, less than -200. If the difference between the fast and slow moving averages does not exceed 200 within the next 1000 milliseconds, this frame is designated as the speech end point. Between frames 33 and 43, the fast moving average is less than the slow moving average, but the difference does not fall below -200, which does not affect the judgment.

[0050] Example 2

[0051] Multilingual simultaneous interpretation agent assistance system in noisy environments (see Figure 2 ), including the following specific modules: data acquisition module, data processing module, sound energy calculation module, moving average calculation module, and speech start and end judgment module;

[0052] Data acquisition module: responsible for obtaining voice data from the input device and passing it to the data processing module.

[0053] Data processing module: pre-processes the collected voice data, such as denoising and normalization, and then divides it into data frames and passes it to the sound energy calculation module.

[0054] Sound energy calculation module: processes the speech signal in each data frame, calculates its sound energy, and passes the result to the moving average calculation module.

[0055] Moving average calculation module: receives data from the sound energy calculation module, calculates the fast and slow moving averages respectively, and passes the results to the speech start and end judgment module.

[0056] Speech start and end judgment module: According to the changes in the moving average value and the dynamically adjusted threshold, it judges the start and end positions of the speech and outputs relevant information or proceeds to the next step of processing.

[0057] S1 Data Frame Division: This system uses the SIP protocol for voice communication and adopts the default SIP packet interval parameter of 20 milliseconds. Therefore, data frames are divided into 20-millisecond intervals to perfectly match SIP. Each point on the horizontal axis in the figure represents a data frame, which will serve as the basic unit for subsequent processing.

[0058] S2 Sound Energy Calculation: This process processes the speech signal in each data frame and calculates its sound energy, which is the mean of the absolute values ​​of the sampled data. This is used for subsequent moving average calculations. Each point on the blue curve in the figure represents the sound energy value for that frame.

[0059] S3 Moving Average Calculation: The new frame coefficients for the fast and slow moving averages are set to 30 and 10, respectively. The fast and slow moving averages are calculated for each frame in real time. In the figure, each point on the gray curve represents the fast moving average for that frame, and each point on the orange curve represents the slow moving average for that frame.

[0060] S4 speech start and end judgment: Determine the start and end of speech based on the difference between the fast and slow moving averages and the dynamically adjusted threshold.

[0061] In the figure, the difference between the fast moving average and the slow moving average of frame 21 is 459, which exceeds the set threshold of 200. This frame is set as the speech start reference point. The differences between the fast moving average and the slow moving average of frames 35, 57, and 82 (-439, -305, and -320, respectively) are all less than -200 and are set as the speech end reference points. However, since the differences between the fast moving average and the slow moving average of frames 48 and 64 (614 and 492, respectively) are both greater than 200 and the intervals with the previous speech end reference points (frames 35 and 57) are all within the pause interval threshold of 1000 milliseconds, the speech end reference points of frames 35 and 57 are invalid, and the speech end point is finally determined to be frame 82.

[0062] It can be seen from the above embodiments that the present invention effectively solves the technical problems of difficult threshold selection, inaccurate single moving average and poor adaptability in traditional methods by introducing a method of fast / slow moving to average value difference, combining fast and slow moving averages, and improving the detection method of adaptability and accuracy. It has higher practical value and broad application prospects.

[0063] Although the specific embodiments of the present invention have been described in detail, those skilled in the art will understand that various modifications and changes can be made to the details based on all the teachings published, and these changes are all within the scope of protection of the present invention. The entire invention is given by the appended claims and any equivalents thereof.

Claims

1. A method for implementing multilingual simultaneous interpretation agent assistance in a noisy environment, characterized in that: The following steps are involved: S1 divides the sampled data frames: determines a time interval value and divides the continuous sampled data of the speech signal into several data frames; S2 calculates sound energy: processes the sampled data in each data frame and calculates its sound energy, that is, the mean of the absolute values ​​of the sampled data; S3 calculates fast and slow moving averages: processes the sound energy in each data frame and calculates fast and slow moving averages respectively; S4 determines the start and end of speech: according to the changes of fast and slow moving averages, the start and end of speech are determined; In step S3, the fast moving average can track the change of sound energy more sensitively through the larger new frame coefficient, while the slow moving average can reflect the change trend of long-term sound energy more smoothly through the smaller new frame coefficient; The new frame coefficient is a value between 0 and 100, excluding endpoints, indicating the proportion of the acoustic moving sound energy of the latest frame of speech data in the moving average value; The calculation formula of moving average is: current frame moving average = new frame coefficient * current frame sound energy + (100-new frame coefficient) * previous frame moving average; In step S4, when the fast moving average value changes from less than (slow moving average value + threshold value) to greater than (slow moving average value + threshold value), it is determined that the sound starts; when the fast moving average value changes from greater than (slow moving average value - threshold value) to less than (slow moving average value - threshold value), it is determined that the sound ends.

2. According to the method of claim 1, which is characterized by: The new frame coefficient is an adjustable parameter, and the new frame coefficients of the fast and slow moving averages can be respectively set to 30 and 10 based on experience.

3. The method for implementing multilingual simultaneous interpretation agent assistance in a noisy environment according to claim 1, characterized in that: The threshold in step S4 is dynamically adjusted to meet the sound detection requirements in different noise environments.

4. A detection device for implementing multi-language simultaneous interpretation seat assistance in a noisy environment, characterized in that: The software or firmware part of the detection device contains a software algorithm for implementing the method described in any one of claims 1 to 3.

5. The detection device according to claim 4, characterized in that: It includes the following specific modules: data acquisition module, data processing module, sound energy calculation module, moving average calculation module, and speech start and end judgment module; The data acquisition module is responsible for acquiring voice data from the input device; The data processing module pre-processes the collected voice data; The sound energy calculation module calculates the sound energy of each data frame; The moving average calculation module calculates fast and slow moving averages; The speech start and end judgment module judges the start and end of speech according to the change of the moving average value.

6. Application of the method according to any one of claims 1 to 3 in multilingual simultaneous interpretation call center seats.

7. Application of the detection device as claimed in claim 4 in a multilingual simultaneous interpretation call center seat.

Images