Post-processing method, device, equipment and storage medium for online endpoint detection

Abstract

The present invention is applicable to the technical field of audio processing, and provides a post-processing method, device, equipment and storage medium for online endpoint detection. The method includes: receiving input of real-time audio, obtaining a current audio segment of a preset cache length, performing online endpoint detection on the current audio segment, obtaining an average probability value of the endpoint detection probability result corresponding to the current audio segment, determining a processing method for the current audio segment according to the average probability value and the current state of a finite state transfer machine, and determining whether to transfer the current state, thereby setting different judgment methods according to different volatility in the audio, reducing the risk of loss of voice that needs to be retained in the audio, and improving the performance and accuracy of online endpoint detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of audio processing, and in particular relates to a post-processing method, device, equipment and storage medium for online endpoint detection. Background Art

[0002] Endpoint detection (Voice Activity Detection, VAD), also known as voice activity detection, aims to distinguish between speech and non-speech areas in audio frames, that is, to accurately locate the start and end points of speech from noisy speech, remove the silent part, remove the noisy part, and find the truly effective content of a speech segment. Endpoint detection can be divided into two scenarios: offline endpoint detection and online endpoint detection. The main task of offline endpoint detection is to obtain the start and end endpoints of a speech segment from a given speech segment. All the information of the speech segment has been obtained during post-processing. The main task of online endpoint detection is to determine in real time whether the audio with a shorter cache length is a speech segment and retain it, with low latency and high real-time requirements.

[0003] Based on the periodic characteristics of speech, endpoint detection usually selects the length of speech frames between 25ms and 50ms. Since the length of speech frames is very short, the accuracy of the results of endpoint detection without post-processing is usually not stable enough, and speech loss is prone to occur. This loss is especially common at the beginning of the speech end, the pause in the middle of the speech, and the end of the speech. Since subsequent speech processing usually includes algorithms such as keyword detection and speech recognition that are greatly affected by speech integrity, endpoint detection post-processing needs to reduce speech loss as much as possible. At present, the common online endpoint detection post-processing method is the threshold method, that is, the judgment result of the buffered audio is averaged and then the threshold value is judged. If it is higher than the threshold value, it is judged as a speech frame, and if it is lower than the threshold value, it is judged as a non-speech frame. This method reduces the instability of the endpoint detection algorithm by smoothing the judgment results of multiple frames of speech segments. It can mainly solve the problem that endpoint detection is easily woken up by sudden noise. However, since the prediction accuracy of the endpoint detection algorithm is usually low at the beginning and end of the speech segment and the pause in the middle of the speech segment, the problem that these parts are more likely to lose the audio frames that need to be retained cannot be solved well by averaging. Summary of the invention

[0004] The purpose of the present invention is to provide a post-processing method, device, equipment and storage medium for online endpoint detection, aiming to solve the problems of poor online endpoint detection performance and high risk of loss of voice to be retained in audio caused by the prior art.

[0005] In one aspect, the present invention provides a post-processing method for online endpoint detection, the method comprising the following steps:

[0006] Receive real-time audio input and obtain the current audio segment of a preset buffer length;

[0007] Performing online endpoint detection on the current audio segment to obtain an average probability value of endpoint detection probability results corresponding to the current audio segment;

[0008] Determine how to process the current audio segment and whether to transfer the current state based on the average probability value and the current state of the finite state transfer machine, wherein the current state includes a non-speech state, a preheating state, a speech state, and a cooling state.

[0009] Preferably, the step of determining the processing method of the current audio segment according to the average probability value and the current state of the finite state transfer machine, and determining whether to transfer the current state includes:

[0010] When the current state of the finite state transfer machine is a non-speech state, comparing the average probability value with a preset first probability threshold;

[0011] When the average probability value is greater than the first probability threshold, retaining the current audio segment, and transferring the current state of the finite state transfer machine to a preheating state;

[0012] When the average probability value is not greater than the first probability threshold, the current audio segment is discarded.

[0013] Preferably, the step of determining the processing method of the current audio segment according to the average probability value and the current state of the finite state transfer machine, and determining whether to transfer the current state, further includes:

[0014] When the current state of the finite state transfer machine is a preheating state, retaining the current audio segment, and comparing the count of a preset first time counter with a preset first time threshold;

[0015] When the count of the first time counter is not less than the first time threshold, calculating a first overall average probability value of all audio segments for which online endpoint detection is performed within a time period of the first time threshold;

[0016] When the calculated first overall average probability value is greater than a preset second probability threshold, the current state of the finite state transfer machine is transferred to a speech state; when the calculated first overall average probability value is not greater than the second probability threshold, the current state of the finite state transfer machine is transferred to a non-speech state.

[0017] Preferably, the step of determining the processing method of the current audio segment according to the average probability value and the current state of the finite state transfer machine, and determining whether to transfer the current state, further includes:

[0018] When the current state of the finite state transfer machine is the speech state, the current audio segment is retained, and when the average probability value is not greater than a preset third probability threshold, the current state of the finite state transfer machine is transferred to the cooling state.

[0019] Preferably, the step of determining the processing method of the current audio segment according to the average probability value and the current state of the finite state transfer machine, and determining whether to transfer the current state, further includes:

[0020] When the current state of the finite state transfer machine is a cooling state, retaining the current audio segment, and comparing the count of a preset second time counter with a preset second time threshold;

[0021] When the count of the second time counter is not less than the second time threshold, calculating a second overall average probability value of all audio segments for which online endpoint detection is performed within a time period of the second time threshold;

[0022] When the calculated second overall average probability value is greater than the preset fourth probability threshold, the current state of the finite state transfer machine is transferred to the speech state; when the second overall average probability value is not greater than the fourth probability threshold, the current state of the finite state transfer machine is transferred to the non-speech state.

[0023] In another aspect, the present invention provides a post-processing device for online endpoint detection, the device comprising:

[0024] An audio segment acquisition unit, used to receive real-time audio input and obtain a current audio segment of a preset buffer length;

[0025] a probability value acquisition unit, configured to perform online endpoint detection on the current audio segment to obtain an average probability value of the endpoint detection probability results corresponding to the current audio segment; and

[0026] A processing mode determination unit is used to determine a processing mode for the current audio segment and whether to transfer the current state according to the average probability value and the current state of the finite state transfer machine, wherein the current state includes a non-speech state, a preheating state, a speech state and a cooling state.

[0027] Preferably, the processing mode determination unit includes:

[0028] A first comparison unit, configured to compare the average probability value with a preset first probability threshold when the current state of the finite state transfer machine is a non-speech state;

[0029] a first processing unit, configured to retain the current audio segment and transfer the current state of the finite state transfer machine to a preheating state when the average probability value is greater than the first probability threshold; and

[0030] The second processing unit is configured to discard the current audio segment when the average probability value is not greater than the first probability threshold.

[0031] Preferably, the processing mode determination unit further includes:

[0032] A second comparison unit is used for retaining the current audio segment and comparing the count of the preset first time counter with the preset first time threshold when the current state of the finite state transfer machine is a preheating state;

[0033] A first probability calculation unit is configured to calculate a first overall average probability value of all audio segments that have undergone online endpoint detection within a time period of the first time threshold when the count of the first time counter is not less than the first time threshold; and

[0034] The third processing unit is used to transfer the current state of the finite state transfer machine to a speech state when the calculated first overall average probability value is greater than a preset second probability threshold, and to transfer the current state of the finite state transfer machine to a non-speech state when the first overall average probability value is not greater than the second probability threshold.

[0035] On the other hand, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps described in the post-processing method for online endpoint detection are implemented.

[0036] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps described in the post-processing method for online endpoint detection are implemented.

[0037] The present invention receives an input of real-time audio, obtains a current audio segment of a preset cache length, performs online endpoint detection on the current audio segment, obtains an average probability value of an endpoint detection probability result corresponding to the current audio segment, determines a processing method for the current audio segment according to the average probability value and a current state of a finite state transfer machine, and determines whether to transfer the current state, thereby setting different judgment methods according to different volatility in the audio, reducing the risk of loss of voice that needs to be retained in the audio, and improving the performance and accuracy of online endpoint detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the implementation of the post-processing method for online endpoint detection provided in the first embodiment of the present invention;

[0039] Figure 2 It is a state diagram of a finite state transfer machine in a post-processing method for online endpoint detection provided in the first embodiment of the present invention;

[0040] Figure 3 is a flow chart of the implementation of the post-processing method for online endpoint detection provided by the second embodiment of the present invention;

[0041] Figure 4 is a structural diagram of a post-processing device for online endpoint detection provided in Embodiment 3 of the present invention; and

[0042] Figure 5 It is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0044] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments:

[0045] Embodiment 1:

[0046] Figure 1 The implementation process of the post-processing method for online endpoint detection provided by the first embodiment of the present invention is shown. For the convenience of description, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0047] In step S101, real-time audio input is received to obtain a current audio segment of a preset buffer length.

[0048] The embodiment of the present invention is applicable to electronic devices with online endpoint detection functions, such as mobile phones, bracelets, tablet computers, portable computers, etc. In the embodiment of the present invention, the audio input is received in real time through the microphone of the electronic device, and the current audio segment of the corresponding length is obtained according to the buffer length of the preset buffer, so as to perform real-time processing on the current audio segment of the buffer length later.

[0049] Preferably, the buffer length is set to 1, thereby improving the real-time performance of processing the current audio segment.

[0050] In step S102, online endpoint detection is performed on the current audio segment to obtain an average probability value of the endpoint detection probability results corresponding to the current audio segment.

[0051] In the embodiment of the present invention, there are n audio frames in the current audio segment with a cache length of 1. After performing online endpoint detection on the current audio segment, endpoint detection probability results P1, ..., P corresponding to the n audio frames in the current audio segment are obtained. n , and the probability results P1,...,P n The cached probability results P1, ..., P are passed to the preset buffer, and the cached probability results P1, ..., P n Calculate the average value to obtain the average probability value P corresponding to the current audio segment = (P1+...+P n ) / n, where P n is the probability result corresponding to the nth frame of audio.

[0052] In step S103, the processing method for the current audio segment and whether to transfer the current state are determined according to the average probability value and the current state of the finite state transition machine, wherein the current state includes a non-speech state, a preheating state, a speech state, and a cooling state.

[0053] In the embodiment of the present invention, according to the different fluctuations of the online endpoint detection for the beginning, middle and end judgments of the non-speech segment and the speech segment in the audio, the finite state transfer machine is divided into four states: non-speech state, warm-up state, speech state and cooling state. That is, the non-speech state represents the non-speech segment part in the audio, the warm-up state represents the beginning part of the speech segment in the audio, the speech state represents the middle part of the speech segment in the audio, and the cooling state represents the ending part of the speech segment in the audio. The transfer paths between the states are as follows: Figure 2As shown: the non-speech state can be transferred to the preheating state, the preheating state can be transferred to the speech state or the non-speech state, the speech state can be transferred to the cooling state, and the cooling state can be transferred to the speech state or the non-speech state. When the finite state transfer machine receives the average probability value output from the buffer, it determines whether to retain or discard the current audio segment according to the average probability value corresponding to the current audio segment and the current state of the finite state transfer machine, and at the same time determines whether to transfer the current state of the finite state transfer machine.

[0054] In an embodiment of the present invention, real-time audio input is received, a current audio segment of a preset cache length is obtained, online endpoint detection is performed on the current audio segment, an average probability value of the endpoint detection probability result corresponding to the current audio segment is obtained, and a processing method for the current audio segment is determined based on the average probability value and the current state of the finite state transfer machine, as well as a determination as to whether to transfer the current state, thereby setting different judgment methods according to different volatility in the audio, reducing the risk of loss of voice that needs to be retained in the audio, and improving the performance and accuracy of online endpoint detection.

[0055] Embodiment 2:

[0056] Figure 3 The implementation process of the post-processing method for online endpoint detection provided by the second embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0057] In step S301, real-time audio input is received to obtain a current audio segment of a preset buffer length.

[0058] In step S302, online endpoint detection is performed on the current audio segment to obtain an average probability value of the endpoint detection probability results corresponding to the current audio segment.

[0059] In the embodiment of the present invention, the specific implementation of step S301-step S302 can refer to the description of step S101-step S102 in embodiment 1, which will not be repeated here.

[0060] In step S303, the current state of the finite state transfer machine is determined.

[0061] In an embodiment of the present invention, when the finite state transfer machine receives the average probability value output from the buffer, it determines the current state of the finite state transfer machine. When the current state is a non-speech state, it jumps to step S304. When the current state is a preheating state, it jumps to step S307. When the current state is a speech state, it jumps to step S310. When the current state is a cooling state, it jumps to step S311.

[0062] In step S304, the average probability value is compared with a preset first probability threshold.

[0063] In an embodiment of the present invention, when the current state of the finite state transfer machine is a non-speech state, the average probability value is compared with a preset first probability threshold. When the average probability value is greater than the preset first probability threshold, step S305 is executed; when the average probability value is not greater than the preset first probability threshold, step S306 is executed.

[0064] In step S305, the current audio segment is retained, and the current state of the finite state transfer machine is transferred to the preheating state.

[0065] In step S306, the current audio segment is discarded.

[0066] In an embodiment of the present invention, when the average probability value is greater than a preset first probability threshold, the current audio segment is retained, that is, the current audio segment cached in the buffer is output, and the current state of the finite state transfer machine is transferred to a preheating state. When the average probability value is less than or equal to the first probability threshold, the current audio segment is discarded, that is, the current audio segment is not output, and the current state of the finite state transfer machine is still a non-speech state.

[0067] Preferably, the first probability threshold can be determined based on the performance of the online endpoint detection algorithm for non-speech noise being misdetected as speech and the performance of speech at the beginning of a speech segment being missed as non-speech, thereby improving the detection performance of online endpoint detection for non-speech.

[0068] In step S307, the current audio segment is retained, and the count of the preset first time counter is compared with the preset first time threshold.

[0069] In an embodiment of the present invention, when the finite state transfer machine is currently in a preheating state, it indicates that the current audio segment is the beginning part of the speech, and the current audio segment is retained, and the count of a preset first time counter is compared with a preset first time threshold, wherein the count of the first time counter is initially set to 0, and when the finite state transfer machine receives the average probability value output from the buffer, the count of the first time counter increases by 1, which can be expressed as t1=t1+1, where t1 is the count of the first time counter.

[0070] In step S308, when the count of the first time counter is not less than the first time threshold, a first overall average probability value of all audio segments for which online endpoint detection is performed within a time period of the first time threshold is calculated.

[0071] In an embodiment of the present invention, when the count of the first time counter is less than the first time threshold, the current state of the finite state transfer machine is still in the preheating state. When the count of the first time counter is greater than or equal to the first time threshold, the count of the first time counter is reset to zero. At the same time, based on the average probability value of all audio segments received by the finite state transfer machine within the time period of the first time threshold, the first overall average probability value of all audio segments that have undergone online endpoint detection within the time period of the first time threshold is calculated.

[0072] In step S309, when the calculated first overall average probability value is greater than the preset second probability threshold, the current state of the finite state transfer machine is transferred to the speech state; when the first overall average probability value is not greater than the second probability threshold, the current state of the finite state transfer machine is transferred to the non-speech state.

[0073] In an embodiment of the present invention, when the calculated first overall average probability value is greater than a preset second probability threshold, the current state of the finite state transfer machine is transferred to a speech state; when the first overall average probability value is less than or equal to the second probability threshold, the current state of the finite state transfer machine is transferred to a non-speech state.

[0074] Preferably, the first time threshold and the second probability threshold can be determined according to the online endpoint detection algorithm based on the performance of non-speech noise being continuously misdetected as speech in a non-speech state and the performance of speech being misdetected as non-speech at the beginning of a speech segment, thereby improving the detection performance of the online endpoint detection for the beginning part of a speech segment.

[0075] In step S310, the current audio segment is retained, and when the average probability value is not greater than a preset third probability threshold, the current state of the finite state transfer machine is transferred to a cooling state.

[0076] In an embodiment of the present invention, when the finite state transfer machine is currently in a speech state, the current audio segment is retained. At the same time, if the average probability value is greater than a preset third probability threshold at this time, the current state of the finite state transfer machine remains in the speech state. If the average probability value is less than or equal to the preset third probability threshold at this time, the current state of the finite state transfer machine is transferred to a cooling state.

[0077] Preferably, the third probability threshold can be determined according to the performance of the online endpoint detection algorithm for misidentifying short pauses in the middle of speech as non-speech, thereby improving the detection performance of the online endpoint detection for short pauses in the middle of speech.

[0078] In step S311, the current audio segment is retained, and the count of the preset second time counter is compared with the preset second time threshold.

[0079] In an embodiment of the present invention, when the finite state transfer machine is currently in a cooling state, it indicates that the current audio segment is the end part of the speech, and the current audio segment is retained, and the count of a preset second time counter is compared with a preset second time threshold, wherein the count of the second time counter is initially set to 0, and when the finite state transfer machine receives the average probability value output from the buffer, the count of the second time counter increases by 1, which can be expressed as t2=t2+1, where t2 is the count of the second time counter.

[0080] In step S312, when the count of the second time counter is not less than the second time threshold, a second overall average probability value of all audio segments for which online endpoint detection is performed within a time period of the second time threshold is calculated.

[0081] In an embodiment of the present invention, when the count of the second time counter is less than the second time threshold, the current state of the finite state transfer machine is still in the cooling state. When the count of the second time counter is greater than or equal to the second time threshold, the count of the second time counter is reset to zero. At the same time, based on the average probability value of all audio segments received by the finite state transfer machine within the time period of the second time threshold, the second overall average probability value of all audio segments that have undergone online endpoint detection within the time period of the second time threshold is calculated.

[0082] In step S313, when the calculated second overall average probability value is greater than the preset fourth probability threshold, the current state of the finite state transfer machine is transferred to the speech state; when the second overall average probability value is not greater than the fourth probability threshold, the current state of the finite state transfer machine is transferred to the non-speech state.

[0083] In an embodiment of the present invention, when the calculated second overall average probability value is greater than a preset fourth probability threshold, the current state of the finite state transfer machine is transferred to a speech state; when the second overall average probability value is less than or equal to the fourth probability threshold, the current state of the finite state transfer machine is transferred to a non-speech state.

[0084] Preferably, the second time threshold and the fourth probability threshold can be determined according to the online endpoint detection algorithm for the performance of speech being continuously misdetected as non-speech in the middle of a speech segment, the performance of speech being misidentified as non-speech at the end of a speech segment, and the performance of non-speech being misidentified as speech after the end of a speech segment, thereby improving the detection performance of the end part of a speech segment by the online endpoint detection.

[0085] In an embodiment of the present invention, according to the different fluctuations of the online endpoint detection in judging the beginning, middle and end of non-speech segments and speech segments, different judgment methods are set for different fluctuations by setting a finite state transition machine including four states: non-speech state, preheating state, speech state, and cooling state, thereby improving the performance and accuracy of the online endpoint detection.

[0086] Embodiment three:

[0087] Figure 4 The structure of the post-processing device for online endpoint detection provided by the third embodiment of the present invention is shown. For the convenience of description, only the part related to the embodiment of the present invention is shown, including:

[0088] The audio segment acquisition unit 41 is used to receive the input of real-time audio and obtain the current audio segment of the preset buffer length;

[0089] A probability value acquisition unit 42 is used to perform online endpoint detection on the current audio segment to obtain an average probability value of the endpoint detection probability results corresponding to the current audio segment; and

[0090] The processing mode determination unit 43 is used to determine the processing mode of the current audio segment and whether to transfer the current state according to the average probability value and the current state of the finite state transfer machine, wherein the current state includes a non-speech state, a preheating state, a speech state and a cooling state.

[0091] Preferably, the processing mode determination unit 43 includes:

[0092] A first comparison unit, used for comparing the average probability value with a preset first probability threshold when the current state of the finite state transfer machine is a non-speech state;

[0093] A first processing unit, configured to retain the current audio segment and transfer the current state of the finite state transfer machine to a preheating state when the average probability value is greater than a first probability threshold; and

[0094] The second processing unit is configured to discard the current audio segment when the average probability value is not greater than the first probability threshold.

[0095] Preferably, the processing mode determination unit 43 further includes:

[0096] A second comparison unit is used for retaining the current audio segment and comparing the count of the preset first time counter with the preset first time threshold when the current state of the finite state transfer machine is the preheating state;

[0097] A first probability calculation unit is configured to calculate a first overall average probability value of all audio segments for which online endpoint detection is performed within a time period of the first time threshold when the count of the first time counter is not less than the first time threshold; and

[0098] The third processing unit is used to transfer the current state of the finite state transfer machine to a speech state when the calculated first overall average probability value is greater than a preset second probability threshold, and to transfer the current state of the finite state transfer machine to a non-speech state when the first overall average probability value is not greater than the second probability threshold.

[0099] Preferably, the processing mode determination unit 43 further includes:

[0100] The fourth processing unit is used to retain the current audio segment when the current state of the finite state transfer machine is the speech state, and transfer the current state of the finite state transfer machine to the cooling state when the average probability value is not greater than a preset third probability threshold.

[0101] Preferably, the processing mode determination unit 43 further includes:

[0102] A third comparing unit, configured to retain the current audio segment and compare the count of the preset second time counter with the preset second time threshold when the current state of the finite state transfer machine is a cooling state;

[0103] A second probability calculation unit is configured to calculate a second overall average probability value of all audio segments for which online endpoint detection has been performed within a time period of the second time threshold when the count of the second time counter is not less than the second time threshold; and

[0104] The fifth processing unit is used to transfer the current state of the finite state transfer machine to the speech state when the calculated second overall average probability value is greater than a preset fourth probability threshold, and to transfer the current state of the finite state transfer machine to the non-speech state when the second overall average probability value is not greater than the fourth probability threshold.

[0105] In the embodiment of the present invention, each unit of the post-processing device for online endpoint detection can be implemented by a corresponding hardware or software unit, and each unit can be an independent software or hardware unit, or can be integrated into a software or hardware unit, which is not intended to limit the present invention. Specifically, the implementation of each unit can refer to the description of the aforementioned method embodiments 1 and 2, and will not be repeated here.

[0106] Embodiment 4:

[0107] Figure 5 The structure of an electronic device provided by a fourth embodiment of the present invention is shown. For the convenience of description, only the parts related to the embodiment of the present invention are shown.

[0108] The electronic device 5 of the embodiment of the present invention includes a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, the steps of the post-processing method of online endpoint detection in the embodiment are implemented, such as Figure 1 Alternatively, when the processor 50 executes the computer program 52, the functions of each unit in the above-mentioned device embodiments are realized, for example Figure 4 The functions of the units 41 to 43 are shown.

[0109] In an embodiment of the present invention, real-time audio input is received, a current audio segment of a preset cache length is obtained, online endpoint detection is performed on the current audio segment, an average probability value of the endpoint detection probability result corresponding to the current audio segment is obtained, and a processing method for the current audio segment is determined based on the average probability value and the current state of the finite state transfer machine, as well as a determination as to whether to transfer the current state, thereby setting different judgment methods according to different volatility in the audio, reducing the risk of loss of voice that needs to be retained in the audio, and improving the performance and accuracy of online endpoint detection.

[0110] The electronic device of the embodiment of the present invention can be a mobile phone, a bracelet, a tablet computer, or a portable computer. The steps implemented when the processor 50 in the electronic device 5 executes the computer program 52 to implement the post-processing method of online endpoint detection can refer to the description of the aforementioned method embodiment, which will not be repeated here.

[0111] Embodiment five:

[0112] In an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned post-processing method embodiment of online endpoint detection are implemented, for example, Figure 1 Alternatively, when the computer program is executed by a processor, the functions of each unit in the above-mentioned device embodiments are realized, for example Figure 4 The functions of the units 41 to 43 are shown.

[0113] In an embodiment of the present invention, real-time audio input is received, a current audio segment of a preset cache length is obtained, online endpoint detection is performed on the current audio segment, an average probability value of the endpoint detection probability result corresponding to the current audio segment is obtained, and a processing method for the current audio segment is determined based on the average probability value and the current state of the finite state transfer machine, as well as a determination as to whether to transfer the current state, thereby setting different judgment methods according to different volatility in the audio, reducing the risk of loss of voice that needs to be retained in the audio, and improving the performance and accuracy of online endpoint detection.

[0114] The computer-readable storage medium of the embodiment of the present invention may include any entity or device or recording medium capable of carrying computer program code, for example, ROM / RAM, magnetic disk, optical disk, flash memory and other memories.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A post-processing method for online endpoint detection, characterized in that: The method comprises the following steps: Receive real-time audio input and obtain the current audio segment of a preset buffer length; Performing online endpoint detection on the current audio segment to obtain an average probability value of the endpoint detection probability results corresponding to the current audio segment, wherein there are n frames of audio in the current audio segment. When performing online endpoint detection on the current audio segment to obtain the average probability value of the endpoint detection probability results corresponding to the current audio segment, performing online endpoint detection on the current audio segment to obtain endpoint detection probability results P1, ..., P respectively corresponding to the n frames of audio in the current audio segment n , and the probability results P1,...,P n The cached probability results P1, ..., P n Calculate the average value to obtain the average probability value P corresponding to the current audio segment = (P1+...+P n ) / n,P n is the probability result corresponding to the nth frame of audio; Determine a processing method for the current audio segment and whether to transfer the current state according to the average probability value and the current state of the finite state transfer machine, wherein the current state includes a non-speech state, a preheating state, a speech state, and a cooling state; The step of determining the processing method of the current audio segment according to the average probability value and the current state of the finite state transfer machine, and determining whether to transfer the current state includes: When the current state of the finite state transfer machine is a non-speech state, comparing the average probability value with a preset first probability threshold; When the average probability value is greater than the first probability threshold, retaining the current audio segment, and transferring the current state of the finite state transfer machine to a preheating state; When the average probability value is not greater than the first probability threshold, the current audio segment is discarded.

2. The method according to claim 1, characterized in that The step of determining the processing method of the current audio segment according to the average probability value and the current state of the finite state transfer machine, and determining whether to transfer the current state, further includes: When the current state of the finite state transfer machine is a preheating state, retaining the current audio segment, and comparing the count of a preset first time counter with a preset first time threshold; When the count of the first time counter is not less than the first time threshold, calculating a first overall average probability value of all audio segments for which online endpoint detection is performed within a time period of the first time threshold; When the calculated first overall average probability value is greater than a preset second probability threshold, the current state of the finite state transfer machine is transferred to a speech state; when the calculated first overall average probability value is not greater than the second probability threshold, the current state of the finite state transfer machine is transferred to a non-speech state.

3. The method according to claim 1, characterized in that The step of determining the processing method of the current audio segment according to the average probability value and the current state of the finite state transfer machine, and determining whether to transfer the current state, further includes: When the current state of the finite state transfer machine is the speech state, the current audio segment is retained, and when the average probability value is not greater than a preset third probability threshold, the current state of the finite state transfer machine is transferred to the cooling state.

4. The method according to claim 1, characterized in that The step of determining the processing method of the current audio segment according to the average probability value and the current state of the finite state transfer machine, and determining whether to transfer the current state, further includes: When the current state of the finite state transfer machine is a cooling state, retaining the current audio segment, and comparing the count of a preset second time counter with a preset second time threshold; When the count of the second time counter is not less than the second time threshold, calculating a second overall average probability value of all audio segments for which online endpoint detection is performed within a time period of the second time threshold; When the calculated second overall average probability value is greater than the preset fourth probability threshold, the current state of the finite state transfer machine is transferred to the speech state; when the second overall average probability value is not greater than the fourth probability threshold, the current state of the finite state transfer machine is transferred to the non-speech state.

5. A post-processing device for online endpoint detection, characterized in that: The device comprises: An audio segment acquisition unit, used to receive real-time audio input and obtain a current audio segment of a preset buffer length; A probability value acquisition unit is used to perform online endpoint detection on the current audio segment to obtain an average probability value of the endpoint detection probability results corresponding to the current audio segment, wherein there are n frames of audio in the current audio segment. When online endpoint detection is performed on the current audio segment to obtain the average probability value of the endpoint detection probability results corresponding to the current audio segment, online endpoint detection is performed on the current audio segment to obtain endpoint detection probability results P1, ..., P respectively corresponding to the n frames of audio in the current audio segment. n , and the probability results P1,...,P n The cached probability results P1, ..., P n Calculate the average value to obtain the average probability value P corresponding to the current audio segment = (P1+...+P n ) / n,P n is the probability result corresponding to the nth frame of audio; and a processing mode determining unit, configured to determine a processing mode for the current audio segment and determine whether to transfer the current state according to the average probability value and the current state of the finite state transfer machine, wherein the current state includes a non-speech state, a preheating state, a speech state, and a cooling state; Wherein, the processing mode determination unit includes: A first comparison unit, configured to compare the average probability value with a preset first probability threshold when the current state of the finite state transfer machine is a non-speech state; a first processing unit, configured to retain the current audio segment and transfer the current state of the finite state transfer machine to a preheating state when the average probability value is greater than the first probability threshold; and The second processing unit is configured to discard the current audio segment when the average probability value is not greater than the first probability threshold.

6. The device according to claim 5, characterized in that The processing mode determination unit further includes: A second comparison unit is used for retaining the current audio segment and comparing the count of the preset first time counter with the preset first time threshold when the current state of the finite state transfer machine is a preheating state; A first probability calculation unit is configured to calculate a first overall average probability value of all audio segments that have undergone online endpoint detection within a time period of the first time threshold when the count of the first time counter is not less than the first time threshold; and The third processing unit is used to transfer the current state of the finite state transfer machine to a speech state when the calculated first overall average probability value is greater than a preset second probability threshold, and to transfer the current state of the finite state transfer machine to a non-speech state when the first overall average probability value is not greater than the second probability threshold.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images