Vad identification method and device, storage medium and computer device

Abstract

The application provides a VAD identification method and device, a storage medium and a computer device. The method comprises the following steps: acquiring an audio signal and a vibration signal synchronized with the audio signal; using a preset speech recognition model to analyze the audio signal to obtain a preliminary identification result; correcting the preliminary identification result based on the vibration signal to obtain a first speech recognition result and a correction parameter used for correcting the preliminary identification result; fusing the audio signal, the first speech recognition result and the correction parameter to obtain a fusion signal; and inputting the fusion signal into a preset VAD identification model to output a second speech recognition result as a VAD identification result. The application can improve the accuracy of VAD identification.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to a VAD recognition method, device, storage medium and computer equipment. Background Technology

[0002] With the continuous development of human-computer interaction technology, the requirements for the accuracy of speech recognition are also increasing. In speech recognition technology, it is necessary to use speech endpoint detection technology to find the start and end points of speech segments in continuous sound signals. Speech endpoint detection technology is also commonly known as voice activity detection (VAD). Therefore, the accuracy of VAD recognition has a direct impact on the accuracy of speech recognition, but current VAD recognition methods still have the problem of inaccurate recognition. Summary of the Invention

[0003] This application provides a VAD identification method, apparatus, storage medium, and computer device, which can improve the accuracy of VAD identification.

[0004] Firstly, this application provides a VAD identification method, the method comprising:

[0005] Acquire an audio signal and a vibration signal synchronized with the audio signal;

[0006] The audio signal is analyzed using a preset speech recognition model to obtain preliminary recognition results;

[0007] Based on the vibration signal, the preliminary recognition result is corrected to obtain a first speech recognition result and correction parameters for correcting the preliminary recognition result;

[0008] The audio signal, the first speech recognition result, and the correction parameters are fused to obtain the fused signal;

[0009] The fused signal is input into a preset VAD recognition model, and the second speech recognition result is output as the VAD recognition result.

[0010] In one embodiment, the step of correcting the preliminary recognition result based on the vibration signal to obtain a first speech recognition result and correction parameters for correcting the preliminary recognition result includes:

[0011] The vibration signal is input into a preset vibration signal model to obtain the correction parameters; wherein, the correction parameters are used to reflect the probability that each frame of the vibration signal and the corresponding audio frame in the audio signal are speech signals;

[0012] Based on the correction parameters, the recognition results of each audio frame in the preliminary recognition result are corrected to obtain the first speech recognition result.

[0013] In one embodiment, the step of inputting the vibration signal to a preset vibration signal model to obtain the correction parameters includes:

[0014] The vibration signal is subjected to noise reduction processing;

[0015] The processed vibration signal is input into the vibration signal model to obtain the correction parameters.

[0016] In one embodiment, the step of correcting the recognition result of each audio frame in the preliminary recognition result based on the correction parameters to obtain the first speech recognition result includes:

[0017] The recognition results of each audio frame in the preliminary recognition result are weighted according to the correction parameters to obtain the first speech recognition result.

[0018] In one embodiment, the step of correcting the recognition result of each audio frame in the preliminary recognition result based on the correction parameters to obtain the first speech recognition result includes:

[0019] The preliminary recognition result is masked according to the correction parameters to obtain the first speech recognition result.

[0020] In one embodiment, fusing the audio signal, the first speech recognition result, and the correction parameters to obtain the fused signal includes:

[0021] The audio signal, the first speech recognition result, and the correction parameters are input into a preset fusion model for feature fusion to obtain the fused signal.

[0022] In one embodiment, the fusion model is a residual network model or a convolutional network model.

[0023] Secondly, this application provides a VAD identification device, comprising:

[0024] A signal acquisition module is used to acquire an audio signal and a vibration signal synchronized with the audio signal;

[0025] The first recognition module is used to analyze the audio signal using a preset speech recognition model to obtain preliminary recognition results;

[0026] A correction module is used to correct the preliminary recognition result based on the vibration signal to obtain a first speech recognition result and correction parameters for correcting the preliminary recognition result;

[0027] The fusion module is used to fuse the audio signal, the first speech recognition result, and the correction parameters to obtain a fused signal;

[0028] The second recognition module is used to input the fused signal into a preset VAD recognition model and output a second speech recognition result as the VAD recognition result.

[0029] Thirdly, this application provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the VAD identification method as described in any of the above embodiments.

[0030] Fourthly, this application provides a computer device, including: one or more processors, and a memory;

[0031] The memory stores computer-readable instructions, and when the one or more processors execute the computer-readable instructions, they perform the steps of the VAD identification method as described in any of the above embodiments.

[0032] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0033] The VAD recognition method, apparatus, storage medium, and computer equipment provided in this application acquire audio signals and synchronous vibration signals, where the vibration signals are triggered when a sound source vibrates. A speech recognition model is used to perform preliminary recognition of the audio signals to obtain preliminary recognition results. Since the preliminary recognition results may contain some errors due to environmental noise, and the vibration signals are entirely based on the sound source's emission, using the vibration signals to correct the preliminary recognition results can effectively suppress the influence of environmental noise, thereby obtaining a first speech recognition result and correction parameters for correcting the preliminary recognition result. However, since not all sound emission from a sound source triggers vibration signals that the acquisition device can collect, the first speech recognition result... There may be over-suppression in the results, that is, the sound of some sound sources is also suppressed. On this basis, the audio signal before the initial recognition is introduced and fused with the first speech recognition result and the correction parameters. The audio signal is used to learn the information suppressed by the incorrect correction of the correction parameters, and to learn the first speech recognition result that has completely suppressed the environmental noise after correction, so as to obtain a fused signal. The fused signal is input into the VAD recognition model for final recognition and outputs the second speech recognition result as the VAD recognition result of the audio signal. This can reduce the impact of environmental noise on the recognition accuracy and avoid the problem of model bypass caused by incorrect correction due to insufficient vibration signal information, thereby improving the accuracy of VAD recognition. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating a VAD identification method in one embodiment;

[0036] Figure 2 This is a flowchart illustrating the steps of correcting the initial recognition result based on the vibration signal to obtain the first speech recognition result and the correction parameters used to correct the initial recognition result in one embodiment.

[0037] Figure 3 This is a structural block diagram of a VAD identification device in one embodiment;

[0038] Figure 4 This is a diagram of the internal structure of a computer device in one embodiment;

[0039] Figure 5 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] The VAD recognition method in this application embodiment can be applied to smart wearable devices equipped with audio signal acquisition devices and vibration signal acquisition devices. The audio signal acquisition device is used to acquire the audio signal of the sound source. For example, the audio signal acquisition device can be a microphone or microphone, which can be used to acquire the voice of the wearer wearing the smart wearable device. The vibration signal acquisition device can be used to acquire the vibration signal generated by the bones or skin when the wearer speaks (for example, by acquiring the vibration signal through a bone conduction microphone, vibration sensor, accelerometer, or inertial measurement device that is close to the face and ear). Since the vibration signal is not affected by environmental noise, it can help reduce the influence of environmental noise during the speech recognition process of the audio signal. That is, the smart wearable device can perform VAD recognition based on the audio signal and the vibration signal.

[0042] In some embodiments, the VAD identification method of this application can also be applied to a terminal or server that is connected to the aforementioned smart wearable device via communication. The smart wearable device sends the collected audio signals and vibration signals to the terminal or server, and the terminal or server performs VAD identification.

[0043] like Figure 1 As shown, this application embodiment provides a VAD identification method, including steps S101-S105, wherein:

[0044] Step S101: Acquire the audio signal and the vibration signal synchronized with the audio signal.

[0045] Among them, the synchronization of vibration signal and audio signal refers to the time synchronization of the collected audio signal and vibration signal.

[0046] In one specific implementation, when a smart wearable device integrating vibration signal acquisition and audio signal acquisition is worn by a wearer, if the smart wearable device is enabled, it can simultaneously acquire vibration signals and audio signals as the wearer speaks.

[0047] Step S102: Use a preset speech recognition model to analyze the audio signal and obtain preliminary recognition results.

[0048] The preliminary identification results are used to characterize non-speech audio frames and speech audio frames in the audio signal. The preliminary identification results can be characterized by the probability that each audio frame is a speech audio frame or a non-speech audio frame, or by identifying each audio frame as a speech audio frame or a non-speech audio frame.

[0049] In one embodiment, the speech recognition model may be a VAD model, a pre-trained neural network model, or other statistical model, and the embodiments of the present invention do not specifically limit this.

[0050] In one embodiment, before parsing the audio signal using a preset speech recognition model, the audio signal can be preprocessed, such as through noise reduction, nonlinear processing, or feature enhancement. The preprocessed audio signal is then input into the preset speech recognition model for parsing to obtain preliminary recognition results. Signal preprocessing makes the information carried by the audio signal input to the speech recognition model more salient, and the feature information to be learned by the model clearer, which helps the speech recognition model converge and improves the accuracy of the preliminary recognition results.

[0051] Step S103: Based on the vibration signal, correct the preliminary recognition result to obtain the first speech recognition result and the correction parameters used to correct the preliminary recognition result.

[0052] Among them, the correction parameters are parameters obtained from the vibration signal and used to correct the preliminary identification results.

[0053] Since vibration signals are signals formed by the vibrations generated when a sound source emits sound, collected by a vibration signal acquisition device, in one specific implementation, when a wearer of a smart wearable device that integrates a vibration signal acquisition device and an audio signal acquisition device speaks, the vibration signal can be formed by the vibrations generated by the wearer's bones or skin, and environmental noise will not be reflected in the vibration signal. Therefore, the correction parameters obtained by the vibration signal collected synchronously with the audio signal can correct the audio signal and suppress interference such as environmental noise in the audio signal.

[0054] Step S104: Fuse the audio signal, the first speech recognition result, and the correction parameters to obtain the fused signal.

[0055] In one specific implementation, the frequency of bone or skin vibration during speech by a wearer of a smart wearable device cannot be too fast, thus the upper limit of the vibration signal frequency is not high. Furthermore, since the wearer's vocal cords only vibrate when producing voiced sounds, and not when producing unvoiced sounds, the vibration signal is difficult to recognize unvoiced sounds during speech. That is, the amount of information contained in the acquired vibration signal is lower than the amount of information contained in the synchronously acquired audio signal. Therefore, the first speech recognition result obtained by correcting the preliminary recognition result based on the vibration signal may exhibit excessive suppression of speech information, meaning that effective vocal information from certain sound sources is also suppressed. Based on this, the audio signal before the preliminary recognition is introduced and fused with the first speech recognition result and correction parameters. The information suppressed by incorrect correction parameters is learned from the initially acquired audio signal, and the first speech recognition result, which has completely suppressed environmental noise after correction, is also learned to obtain a fused signal. This fused signal is used for the final VAD recognition. Step S105: The fused signal is input into a preset VAD recognition model, and the second speech recognition result is output as the VAD recognition result.

[0056] The first speech recognition result is obtained by introducing vibration signals. Since the first speech recognition result has been corrected by the correction parameters, it has a high degree of suppression of environmental noise interference. According to the difference in environmental noise, the feature quantity contained in the first speech recognition result corresponding to each audio signal will inevitably differ from the feature quantity in the final output VAD recognition result. In the process of continuous recognition using the VAD recognition model, a vibration signal synchronized with the currently recognized audio signal is introduced for correction each time, which can effectively avoid the problem of overfitting of the VAD recognition model after long-term recognition learning.

[0057] The VAD recognition method provided in this application acquires an audio signal and a vibration signal synchronized with it. The vibration signal is the signal triggered when the sound source vibrates. A speech recognition model is used to perform preliminary recognition on the audio signal to obtain a preliminary recognition result. Since the preliminary recognition result may contain some errors caused by environmental noise, and the vibration signal is entirely based on the sound source's emission, using the vibration signal to correct the preliminary recognition result can effectively suppress the influence of environmental noise. This yields a first speech recognition result and correction parameters used to correct the preliminary recognition result. However, since not all sounds emitted by the sound source trigger the vibration signal that the acquisition device can collect, the first speech recognition result may exhibit over-suppression, i.e., for some... The sound source's emission was also suppressed. Based on this, the audio signal before the initial recognition was introduced and fused with the first speech recognition result and correction parameters. The audio signal was used to learn the information suppressed by the incorrect correction parameters and to learn the first speech recognition result that had completely suppressed environmental noise after correction. The fused signal was then input into the VAD recognition model for final recognition and output as the second speech recognition result as the VAD recognition result of the audio signal. This not only reduces the impact of environmental noise on recognition accuracy but also avoids the problem of model bypass caused by incorrect correction due to insufficient vibration signal information. On the other hand, it also avoids overfitting of the VAD recognition model, thus improving the accuracy of VAD recognition from multiple directions.

[0058] like Figure 2 As shown, in one embodiment, the step of correcting the preliminary recognition result based on the vibration signal to obtain a first speech recognition result and correction parameters for correcting the preliminary recognition result includes:

[0059] Step S201: Input the vibration signal into the preset vibration signal model to obtain the correction parameters.

[0060] The correction parameter is used to reflect the probability that each frame of the vibration signal and the corresponding audio frame in the audio signal are speech signals.

[0061] Step S202: Based on the correction parameters, the recognition results of each audio frame in the preliminary recognition results are corrected to obtain the first speech recognition result.

[0062] Based on each frame of the vibration signal, the corresponding audio frame in the audio signal is determined. The recognition result of each audio frame in the initial recognition result of the audio signal is then corrected using correction parameters to obtain the first speech recognition result. In one embodiment, the correction parameters can be a probability sequence, probability curve, etc.

[0063] In this embodiment, based on the synchronization of vibration and audio signals, a correction parameter, obtained from the vibration signal and reflecting the probability that each frame of the vibration signal and its corresponding audio frame in the audio signal are speech signals, is used to correct the preliminary recognition result. Since the vibration signal does not contain interference information such as environmental noise, the correction parameter is equivalent to the recognition result obtained from analyzing a signal without environmental noise. By using the correction signal to correct the recognition result of each frame of the vibration signal and its corresponding audio frame in the audio signal in the preliminary recognition result, misrecognition caused by interference information such as environmental noise not contained in the vibration signal can be suppressed. In the subsequent audio signal recognition process, the correction parameter can be used as a weighting coefficient or other parameters for feature extraction, or the correction parameter and the preliminary recognition result can be jointly probabilistically calculated to obtain the first speech recognition result. While the reliability of suppressing interference such as environmental noise in audio signals by correcting parameters is relatively high, due to the characteristics of the signals acquired by the vibration signal acquisition device, in cases where the frequency exceeds the range or the wearer's voice does not produce vibration, these signal frames in the vibration signal may be identified as non-speech signals, leading to certain misjudgments. This results in the initial recognition result being oversuppressed. To address such misjudgments, the oversuppressed information is learned from the audio signal by combining the vibration signal with the audio signal, thereby overcoming the misjudgments caused by the vibration signal.

[0064] In one embodiment, the correction is performed by weighting the recognition results of each audio frame in the preliminary recognition result according to the correction parameters to obtain a first speech recognition result. In this embodiment, the weighting process can be either using the correction parameters as weight coefficients to obtain weighted speech features, or it can be based on the correction parameters to correct the probability that each audio frame in the preliminary recognition result is a speech feature.

[0065] In one embodiment, the correction method involves masking the preliminary recognition result based on correction parameters to obtain a first speech recognition result. Masking, or masking process, can be used to achieve speech separation. In this embodiment, the correction parameters reflect the probability that each audio frame in the preliminary recognition result is a speech audio frame. Audio frames with a preset probability are identified as speech audio frames, while those without are considered not to be speech audio frames. Based on this, a mask is obtained for masking, such as a binary mask where 1 corresponds to a speech audio frame and 0 corresponds to a non-speech audio frame. The preliminary recognition result is then masked based on the binary mask, resulting in the first speech recognition result, which is the speech audio frame extracted after masking.

[0066] In one embodiment, the step of inputting the vibration signal to a preset vibration signal model to obtain correction parameters includes:

[0067] Noise reduction processing is applied to the vibration signal;

[0068] The processed vibration signal is input into the vibration signal model to obtain the correction parameters.

[0069] Vibration signal acquisition devices generate signals with some interference during the acquisition process. Depending on the vibration signal acquisition device used, selecting an appropriate noise reduction method to preprocess the vibration signal and inputting the preprocessed signal into the vibration signal model for identification can improve the accuracy of identification. In some embodiments, noise reduction of the vibration signal can be achieved by selecting a filtering algorithm suitable for the vibration signal acquisition device to filter out the frequency components that are noise in the vibration signal acquisition device.

[0070] In one embodiment, the step of parsing the audio signal using a preset speech recognition model to obtain a preliminary recognition result includes:

[0071] The audio signal is subjected to noise reduction processing;

[0072] The noise-reduced audio signal is input into the speech recognition model to obtain the preliminary recognition result.

[0073] For certain interference signals in the audio signal, which differ significantly from the speech signal, noise reduction can be performed during preprocessing. This can be done by using a linear filter or spectral subtraction to reduce noise, thereby reducing interference in the signal input to the speech recognition model and improving recognition accuracy.

[0074] In one embodiment, the fused audio signal, the first speech recognition result, and the correction parameters are used to obtain the fused signal, including:

[0075] The audio signal, the first speech recognition result, and correction parameters are input into a preset fusion model for feature fusion to obtain a fused signal. The fusion model is used to learn features from the corrected first speech recognition result and to extract over-suppressed features from the audio signal after correction, and then perform fusion learning. During feature fusion, correction parameters are needed to determine the suppressed parts in the first speech recognition result, enabling targeted feature learning from the audio signal to further extract over-suppressed vocal information from the audio signal and ultimately obtain a higher-precision VAD recognition result. Therefore, fusion learning requires simultaneous input of the audio signal, the first speech recognition result, and correction parameters into the fusion model. In one embodiment, the fusion model can be a residual network model or a convolutional network model.

[0076] In one embodiment, the audio signal input to the fusion model is first subjected to noise reduction processing, and the processed audio signal, together with the first speech recognition result and correction parameters, is input into the fusion model for feature fusion.

[0077] For certain interference signals in the audio signal, which differ significantly from the speech signal, they can be removed during preprocessing through noise reduction, such as by using a linear filter or spectral subtraction to reduce noise, thereby reducing interference in the fused signal and improving the accuracy of recognition.

[0078] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0079] The VAD identification device provided in the embodiments of this application is described below. The VAD identification device described below can be referred to in correspondence with the VAD identification method described above.

[0080] like Figure 3 As shown, this application provides a VAD identification device 300, including:

[0081] The signal acquisition module 301 is used to acquire audio signals and vibration signals synchronized with the audio signals;

[0082] The first recognition module 302 is used to analyze the audio signal using a preset speech recognition model to obtain a preliminary recognition result;

[0083] The correction module 303 is used to correct the preliminary recognition result based on the vibration signal to obtain the first speech recognition result and correction parameters for correcting the preliminary recognition result;

[0084] The fusion module 304 is used to fuse the audio signal, the first speech recognition result, and the correction parameters to obtain the fused signal.

[0085] The second recognition module 305 is used to input the fused signal into a preset VAD recognition model and output a second speech recognition result as the VAD recognition result.

[0086] In one embodiment, the correction module includes:

[0087] A vibration signal recognition unit is used to input the vibration signal into a preset vibration signal model to obtain the correction parameters; wherein, the correction parameters are used to reflect the probability that each frame of the vibration signal and the corresponding audio frame in the audio signal are speech signals;

[0088] The correction unit is used to correct the recognition result of each audio frame in the preliminary recognition result based on the correction parameters, so as to obtain the first speech recognition result.

[0089] In one embodiment, the vibration signal identification unit is further configured to perform the following steps:

[0090] The vibration signal is subjected to noise reduction processing;

[0091] The processed vibration signal is input into the vibration signal model to obtain the correction parameters.

[0092] In one embodiment, the correction unit is configured to perform the following steps:

[0093] The recognition results of each audio frame in the preliminary recognition result are weighted according to the correction parameters to obtain the first speech recognition result.

[0094] In one embodiment, the correction unit is configured to perform the following steps:

[0095] The preliminary recognition result is masked according to the correction parameters to obtain the first speech recognition result.

[0096] In one embodiment, the first identification module includes:

[0097] The first noise reduction unit is used for noise reduction processing of the audio signal;

[0098] The speech recognition unit is used to input the noise-reduced audio signal into the speech recognition model to obtain the preliminary recognition result.

[0099] In one embodiment, the fusion module includes:

[0100] The second noise reduction unit is used for noise reduction processing of the audio signal;

[0101] The fusion learning unit is used to input the audio signal, the first speech recognition result and the correction parameters into a preset fusion model to perform feature fusion and obtain the fused signal.

[0102] The division of modules in the above-described VAD identification device is merely illustrative. In other embodiments, the VAD identification device can be divided into different modules as needed to complete all or part of the functions of the VAD identification device. Each module in the above-described VAD identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0103] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps:

[0104] Acquire an audio signal and a vibration signal synchronized with the audio signal;

[0105] The audio signal is analyzed using a preset speech recognition model to obtain preliminary recognition results;

[0106] Based on the vibration signal, the preliminary recognition result is corrected to obtain a first speech recognition result and correction parameters for correcting the preliminary recognition result;

[0107] The audio signal, the first speech recognition result, and the correction parameters are fused to obtain the fused signal;

[0108] The fused signal is input into a preset VAD recognition model, and the second speech recognition result is output as the VAD recognition result.

[0109] In one embodiment, the computer-readable instructions, when executed by the processor, further perform the following steps:

[0110] The vibration signal is input into a preset vibration signal model to obtain the correction parameters; wherein, the correction parameters are used to reflect the probability that each frame of the vibration signal and the corresponding audio frame in the audio signal are speech signals;

[0111] Based on the correction parameters, the recognition results of each audio frame in the preliminary recognition result are corrected to obtain the first speech recognition result.

[0112] In one embodiment, the computer-readable instructions, when executed by the processor, further perform the following steps:

[0113] The vibration signal is subjected to noise reduction processing;

[0114] The processed vibration signal is input into the vibration signal model.

[0115] In one embodiment, the computer-readable instructions, when executed by the processor, further perform the following steps:

[0116] The recognition results of each audio frame in the preliminary recognition result are weighted according to the correction parameters to obtain the first speech recognition result.

[0117] In one embodiment, the computer-readable instructions, when executed by the processor, further perform the following steps:

[0118] The preliminary recognition result is masked according to the correction parameters to obtain the first speech recognition result.

[0119] In one embodiment, the computer-readable instructions, when executed by the processor, further perform the following steps:

[0120] The audio signal, the first speech recognition result, and the correction parameters are input into a preset fusion model for feature fusion to obtain the fused signal.

[0121] In one embodiment, this application also provides a computer device storing computer-readable instructions, wherein when the one or more processors execute the computer-readable instructions, they perform the following steps:

[0122] Acquire an audio signal and a vibration signal synchronized with the audio signal;

[0123] The audio signal is analyzed using a preset speech recognition model to obtain preliminary recognition results;

[0124] Based on the vibration signal, the preliminary recognition result is corrected to obtain a first speech recognition result and correction parameters for correcting the preliminary recognition result;

[0125] The audio signal, the first speech recognition result, and the correction parameters are fused to obtain the fused signal;

[0126] The fused signal is input into a preset VAD recognition model, and the second speech recognition result is output as the VAD recognition result.

[0127] In one embodiment, the processor also performs the following steps when executing computer-readable instructions:

[0128] The vibration signal is input into a preset vibration signal model to obtain the correction parameters; wherein, the correction parameters are used to reflect the probability that each frame of the vibration signal and the corresponding audio frame in the audio signal are speech signals;

[0129] Based on the correction parameters, the recognition results of each audio frame in the preliminary recognition result are corrected to obtain the first speech recognition result.

[0130] In one embodiment, the processor also performs the following steps when executing computer-readable instructions:

[0131] The vibration signal is subjected to noise reduction processing;

[0132] The processed vibration signal is input into the vibration signal model.

[0133] In one embodiment, the processor also performs the following steps when executing computer-readable instructions:

[0134] The recognition results of each audio frame in the preliminary recognition result are weighted according to the correction parameters to obtain the first speech recognition result.

[0135] In one embodiment, the processor also performs the following steps when executing computer-readable instructions:

[0136] The preliminary recognition result is masked according to the correction parameters to obtain the first speech recognition result.

[0137] In one embodiment, the processor also performs the following steps when executing computer-readable instructions:

[0138] The audio signal, the first speech recognition result, and the correction parameters are input into a preset fusion model for feature fusion to obtain the fused signal.

[0139] In one embodiment, schematically, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a VAD (Visual ID) identification method.

[0140] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a VAD (Visual Access Detection) method. The display screen can be an LCD screen or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0141] Those skilled in the art will understand that Figure 4 , Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0143] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0144] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0145] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0146] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A VAD identification method, characterized in that, The method includes: Acquire an audio signal and a vibration signal synchronized with the audio signal; The audio signal is analyzed using a preset speech recognition model to obtain preliminary recognition results; Based on the vibration signal, the preliminary recognition result is corrected to obtain a first speech recognition result and correction parameters for correcting the preliminary recognition result; The audio signal, the first speech recognition result, and the correction parameters are fused to obtain the fused signal; The fused signal is input into a preset VAD recognition model, and the second speech recognition result is output as the VAD recognition result. The step of correcting the preliminary recognition result based on the vibration signal to obtain a first speech recognition result and correction parameters for correcting the preliminary recognition result includes: The vibration signal is input into a preset vibration signal model to obtain the correction parameters; wherein, the correction parameters are used to reflect the probability that each frame of the vibration signal and the corresponding audio frame in the audio signal are speech signals; Based on the correction parameters, the recognition results of each audio frame in the preliminary recognition result are corrected to obtain the first speech recognition result.

2. The VAD identification method according to claim 1, characterized in that, The step of inputting the vibration signal into a preset vibration signal model to obtain the correction parameters includes: The vibration signal is subjected to noise reduction processing; The processed vibration signal is input into the vibration signal model to obtain the correction parameters.

3. The VAD identification method according to claim 1 or 2, characterized in that, The step of correcting the recognition result of each audio frame in the preliminary recognition result based on the correction parameters to obtain the first speech recognition result includes: The recognition results of each audio frame in the preliminary recognition result are weighted according to the correction parameters to obtain the first speech recognition result.

4. The VAD identification method according to claim 1 or 2, characterized in that, The step of correcting the recognition result of each audio frame in the preliminary recognition result based on the correction parameters to obtain the first speech recognition result includes: The preliminary recognition result is masked according to the correction parameters to obtain the first speech recognition result.

5. The VAD identification method according to any one of claims 1 to 2, characterized in that, The process of fusing the audio signal, the first speech recognition result, and the correction parameters to obtain the fused signal includes: The audio signal, the first speech recognition result, and the correction parameters are input into a preset fusion model for feature fusion to obtain the fused signal.

6. The VAD identification method according to claim 5, characterized in that, The fusion model is either a residual network model or a convolutional network model.

7. A VAD identification device, characterized in that, include: A signal acquisition module is used to acquire an audio signal and a vibration signal synchronized with the audio signal; The first recognition module is used to analyze the audio signal using a preset speech recognition model to obtain preliminary recognition results; A correction module is used to correct the preliminary recognition result based on the vibration signal to obtain a first speech recognition result and correction parameters for correcting the preliminary recognition result; The fusion module is used to fuse the audio signal, the first speech recognition result, and the correction parameters to obtain a fused signal; The second recognition module is used to input the fused signal into a preset VAD recognition model and output a second speech recognition result as the VAD recognition result; The correction module includes: A vibration signal recognition unit is used to input the vibration signal into a preset vibration signal model to obtain the correction parameters; wherein, the correction parameters are used to reflect the probability that each frame of the vibration signal and the corresponding audio frame in the audio signal are speech signals; The correction unit is used to correct the recognition result of each audio frame in the preliminary recognition result based on the correction parameters, so as to obtain the first speech recognition result.

8. A storage medium, characterized in that: The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the VAD identification method as described in any one of claims 1 to 6.

9. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions, and when the one or more processors execute the computer-readable instructions, they perform the steps of the VAD identification method as described in any one of claims 1 to 6.

Images