An audio processing method and apparatus

Abstract

The application provides an audio processing method and device, the method comprising: collecting audio data; storing the audio data into a recording queue, wherein the recording queue is a first-in first-out queue; taking out a current audio frame from the recording queue; determining that a silence detection result of the current audio frame is a non-silence audio frame and a global state is a silence state; determining that there are a first frame number of continuous non-silence audio frames; copying an audio frame of a silence detection queue to a tail of a recognition queue, wherein the silence detection queue is a circular queue and the recognition queue is a first-in first-out queue.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech recognition, and in particular to an audio processing method and device. BACKGROUND

[0002] In existing voice interaction scenarios, such as a car voice interaction scenario, user audio data needs to be subjected to speech recognition, and then corresponding voice services are provided for the user.

[0003] In the speech recognition process, a large part of the time is spent in silence, i.e., the user does not speak. The existence of silence consumes the computing resources of speech recognition. However, the existing solution is to first detect non-silence, and then start the recognition program to perform recognition. SUMMARY

[0004] In a first aspect, an embodiment of the present application provides an audio processing method, comprising:

[0005] acquiring audio data;

[0006] storing the audio data into a recording queue, wherein the recording queue is a first-in-first-out queue;

[0007] taking out a current audio frame from the recording queue;

[0008] determining that a silence detection result of the current audio frame is a non-silence audio frame and a global state is a silence state;

[0009] determining that there are a first frame number of non-silence audio frames in succession;

[0010] copying audio frames of a silence detection queue to a recognition queue, wherein the silence detection queue is a circular queue, and the recognition queue is a first-in-first-out queue.

[0011] In some embodiments, the method further comprises:

[0012] inserting the current audio frame into the recognition queue, wherein the current audio frame is located after the copied audio frames of the silence detection queue in the recognition queue.

[0013] In some embodiments, the number of audio frames in the copied silence detection queue is greater than or equal to the first frame number.

[0014] In some embodiments, the method further comprises:

[0015] determining that the global state is a non-silence state, and inserting the current audio frame into the recognition queue.

[0016] In some embodiments, the method further comprises:

[0017] updating the global state to a non-silence state.

[0018] In some embodiments, the method further comprises:

[0019] determining that the current audio frame is a silence audio frame based on the silence detection result, and inserting the current audio frame into the silence detection queue.

[0020] In some embodiments, the silence detection queue comprises a plurality of data spaces connected in a loop, the current audio frame is inserted into a data space pointed to by a tail pointer of the silence detection queue, and the tail pointer moves to a data space pointed to by a head pointer of the silence detection queue along a data update direction of the silence detection queue.

[0021] In some embodiments, the method further comprises:

[0022] determining that the global state is a non-silence state, determining that there are a continuous second number of silence audio frames, and updating the global state to a silence state.

[0023] In some embodiments, the method further comprises:

[0024] real-time recognizing the audio frames in the recognition queue.

[0025] In a second aspect, the embodiments of the present application further provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the audio processing method according to any one of the first aspect when executing the program.

[0026] In a third aspect, the embodiments of the present application further provide a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the audio processing method according to any one of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0028] Figure 1 The application environment diagram in which the audio processing method provided by an embodiment of the present application can run is shown in FIG. 1;

[0029] Figure 2 The flowchart of the audio processing method provided by an embodiment of the present application is shown in FIG. 2;

[0030] Figure 3An interaction diagram of a queue of an audio processing method provided by an embodiment of the present application;

[0031] Figure 4 A scene diagram of data update of a silence detection queue provided by an embodiment of the present application;

[0032] Figure 5 A scene diagram of data update of a silence detection queue provided by an embodiment of the present application;

[0033] Figure 6 A message timing diagram of three tasks of an audio processing method provided by an embodiment of the present application;

[0034] Figure 7 A flow diagram of a silence detection task in an audio processing method provided by an embodiment of the present application;

[0035] Figure 8 A structure diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the present application will be described in detail below with the accompanying drawings of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0037] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second" are usually a category, not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally represents a "or" relationship between the front and rear associated objects.

[0038] In the current vehicle voice interaction and mobile terminal (such as mobile phone) interaction scenarios, such as car voice interaction scenarios, it is necessary to perform voice recognition on the audio data of the user, and then provide corresponding voice services for the user. A large part of the time in the voice recognition process is silent, i.e. silence. The existence of silence consumes the computing resources of voice recognition, but the existing scheme is to first perform silence detection on the audio data of the user, and when the end of silence is detected, the recognition program is started to recognize the subsequent audio data. This way often has the phenomenon of losing frames of head data and tail data.

[0039] Therefore, the embodiment of the present application provides an audio processing method, by setting three independent tasks (recording task, silence detection task and identification task) and two global states (silence state and non-silence state), and the message circulation between the three tasks is realized through a queue, so that the boundary of the silence state change will not appear head data and tail data frame loss phenomenon.

[0040] The audio processing method provided by the embodiment of the present application can be applied to an application environment as shown in the following figure. Figure 1 As shown in the following figure, the application environment provided by the audio processing method of the embodiment of the present application can run. Figure 1 As shown in the following figure, the application environment provided by the audio processing method of the embodiment of the present application can run. Figure 1 The application environment includes a terminal 110 and a server 120, and the terminal 110 and the server 120 communicate through a network. The communication network can be a wireless communication network or a wired communication network. The number of terminals and servers is not limited. The wireless communication network can include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The wired communication network can include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network.

[0041] In some embodiments, the terminal 110 (terminal device) includes various handheld devices, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem with wireless communication functions, such as mobile phones, tablets, desktop notebooks, and smart devices that can run applications, including central control consoles of smart cars, etc. It can specifically refer to a user equipment (UE), an access terminal, a user unit, a user station, a mobile station, a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent or a user device. The terminal device can also be a satellite phone, a cellular phone, a smart phone, a wireless data card, a wireless modem, a machine type communication device, which can be a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication functions, a computing device or other processing devices connected to a wireless modem, a vehicle-mounted device or a wearable device, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical treatment, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a terminal device in 5G network or future communication network, etc. The terminal can be powered by a battery, and can also be attached to a power supply system of a vehicle or a ship and powered by the power supply system of the vehicle or the ship. The power supply system of the vehicle or the ship can also charge the battery of the terminal to prolong the communication time of the terminal.

[0042] The server 120 can be implemented by an independent server or a server cluster composed of multiple servers.

[0043] It should be noted that the implementation of the method in the present application can be directly completed on the terminal 110, directly completed on the server 120, or completed on the server 120 and then sent to the terminal 110 by the server 120.

[0044] Figure 2 The flowchart of the audio processing method provided by the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the method includes the following steps.Figure 2 As shown in FIG. 1, an audio processing method is provided, to which the audio processing method is applied to a terminal in FIG. 1. Figure 1 As shown in FIG. 2, the method includes the following steps: step 210, step 220, step 230 to step 260. The method flow steps are only used as one possible implementation of the present application.

[0045] In step 210, audio data is collected.

[0046] In the embodiment of the present application, the audio data of the user can be obtained through the vehicle-mounted voice system. It should be noted that in different scenarios, the user starts the corresponding application program (Application, APP) according to the application demand, such as the navigation program and the music playing program, and then obtains the corresponding audio data through the application program. For example, after the user starts the navigation program, the user asks for the route of the navigation destination, so that the vehicle-mounted voice system obtains the audio data and then performs voice interaction with the user.

[0047] In step 220, the audio data is stored in a recording queue.

[0048] It should be noted that the recording queue in the embodiment is a first input first output (FIFO) queue, and the size of the queue is fixed, as shown in FIG. 3. Figure 3 As shown in FIG. 3, when performing a recording task, new audio data is placed in the space pointed to by the tail pointer of the recording queue, and when performing a subsequent silence detection task, the audio data is taken from the position pointed to by the head pointer of the recording queue. When the head pointer (head) and the tail pointer (tail) overlap, it indicates that there is no audio data in the recording queue, and the subsequent silence detection task is paused until new audio data is stored. When the recording queue is full, the new audio data is blocked until there is a free space in the recording queue.

[0049] Further, in an embodiment, the audio data is stored in the recording queue frame by frame. Usually, the frame length of the audio frame is 10 ms, and one sample data is 16-bit floating point. The audio data stored in the recording queue can be floating point or fixed point.

[0050] In step 230, a current audio frame is taken out from the recording queue.

[0051] The current audio frame refers to the audio frame pointed to by the current head pointer in the recording queue.

[0052] As shown in FIG. 4, the current audio frame is "a1" pointed to by the head pointer. After "a1" is taken out, the head pointer moves one bit backward and points to "a2". Figure 3

[0053] ​Step 240, determining that the silence detection result of the current frame is a non-silence audio frame, and the global state is a silence state;

[0054] In the execution of the silence detection task, one frame of data, i.e. 10ms of data, is generally taken out at a time, each sample point of a single channel is 16 bits, and the sampling rate is generally 16000hz. Therefore, one frame of data includes 160 single-precision floating-point numbers.

[0055] In this embodiment, FBANK speech feature data can be calculated from the audio data as the input of the subsequent acoustic model. By inputting the FBANK feature into the acoustic model, the mapping of the FBANK feature to the character is realized through the acoustic model, and the result output by the acoustic model is sequentially input into the full connection layer and the softmax function (or sigmoid function). Therefore, the probability value of each frame can be obtained. In this embodiment, a small model (i.e. the acoustic model) is used to realize the silence detection. Since the small model has small calculation amount and small storage resource occupation, it will not occupy too much power consumption even if it is continuously running.

[0056] After obtaining the probability value of each frame, the probability value of each frame is compared with a preset threshold value. If the probability value exceeds the preset threshold value, it is determined that the silence detection result of the frame is a non-silence audio frame, otherwise it is a silence audio frame.

[0057] In this embodiment, the global state is updated according to the silence detection result of the audio frame. Figure 6 After determining the silence detection result of the current audio frame, whether the current audio frame needs to be recognized is determined by combining the global state at the moment before the current audio frame is processed. For example, if the global state is a non-silence state and the current audio frame is a non-silence state, it indicates that the current audio frame probably includes audio data of human voice, and the current audio frame is inserted into the recognition queue. For another example, if the global state is a silence state and the current audio frame is a silence state, it indicates that the current audio frame probably does not include audio data of human voice, and the current audio frame is inserted into the silence detection queue.

[0058] In one example, the global state can be updated according to the silence detection result of the audio frame in a period of time, wherein the global state includes a silence state and a non-silence state. If the global state is a non-silence state before the current audio frame is processed, it indicates that there is a continuous non-silence frame before the current audio frame. If the global state is a silence state before the current audio frame is processed, it indicates that there is a continuous silence frame before the current audio frame.

[0059] Step 250, determining that there is a continuous first frame number of non-silence audio frames;

[0060] It should be noted that under the influence of environmental noise and other factors, the silence detection result of part of the non-human voice audio data may also be a non-silence audio frame, therefore, in the embodiment, when the global state is detected to be a silence state, but the current audio frame is a non-silence audio frame, it is necessary to determine whether there is a continuous first frame number of non-silence audio frames. In the case that the current audio frame is a non-silence audio frame, the global state is a silence state, and there is still a continuous first frame number of non-silence audio frames, it can be indicated that the collected audio data has changed state.

[0061] In one example, the continuous first frame number of non-silence audio frames includes the current audio frame, for example, under the influence of the user's speaking habit, the user may speak slowly and the tail sound is weak in some scenarios, therefore, when the user is far away from the audio collection device, the detection result of the audio frame of the tail sound part during the user's speaking may be a silence audio frame, in order to avoid the loss of part of the human voice audio data caused by the above factors, the continuous first frame number of non-silence audio frames includes the current audio frame is set in the embodiment.

[0062] For example, for an audio frame with a frame length of 10 ms, since usually it takes 100-300 ms for a person to speak a word, the first frame number can be set to 10 frames, then when it is detected that the silence detection results of the audio frames from the (N-10) th frame to the N th frame (that is, the current frame) are all non-silence audio frames, it indicates that the audio frames from the (N-10) th frame to the N th frame can probably identify the first word spoken by the user.

[0063] In another example, the continuous first frame number of non-silence audio frames does not include the current audio frame, for example, in some scenarios, the user may output some invalid voice data intermittently during the process of conveying voice instructions, such as “Hi… hmm (hesitation tone) … play a song” or “Hi…”, in order to avoid the identification task of a large amount of invalid audio data (such as “Hi”, “hmm”) caused by the above factors, the continuous first frame number of non-silence audio frames does not include the current audio frame is set in the embodiment.

[0064] For example, when it is detected that the silence detection results of the audio frames from the (N-10) th frame to the N th frame are all non-silence audio frames, it indicates that there is probably human voice audio data, then if it is detected that the (N+1) th frame (that is, the current frame) is also a non-silence audio frame, it indicates that the audio frames from the (N-10) th frame to the N th frame include valid audio data, otherwise, it indicates that the audio frames from the (N-10) th frame to the N th frame do not include valid audio data.

[0065] In step 260, the audio frames in the silence detection queue are copied to the identification queue, wherein the silence detection queue is a circular queue and the identification queue is a first-in-first-out queue.

[0066] In the embodiment, the silence detection queue is a Ring Buffer queue, as shown in Figures 3 to 5 The silence detection queue can be regarded as a circular buffer, which is divided into N data spaces of equal size, each of which can store a frame of voice data. In the silence detection queue, there are two pointers: a head pointer and a tail pointer.

[0067] As shown in Figure 4 , the data update direction of the silence detection queue is clockwise. When the silence detection queue is empty, the head pointer and the tail pointer point to the same data space. When an audio frame “b1” is added to the silence detection queue, the audio frame “b1” is added to the data space pointed to by the tail pointer, and the tail pointer moves one data space clockwise… and so on, until the head pointer and the tail pointer are connected, at which time the silence detection queue is full for the first time.

[0068] Further, as shown in Figure 5 , when the silence detection queue is full, if the silence detection queue allows the previous data to be overwritten, the oldest data will be overwritten when new data is continuously inserted. For example, when the silence detection queue is full for the first time, the audio frames in the silence detection queue from the head to the tail are “b1”, “b2”, “b3”, “b4”, “b5”, and “b6”. When “b7” needs to be enqueued, “b7” is inserted into the data space pointed to by the current tail pointer, and the tail pointer moves one position backward to point to the same data space as the head pointer. After “b8”, “b9”, and “b10” are enqueued in turn, the oldest audio frames “b1”, “b2”, and “b3” are overwritten by “b8”, “b9”, and “b10” respectively. Therefore, the latest audio frame data in the silence detection queue from the head to the tail are “b4”, “b5”, “b6”, “b7”, “b8”, and “b10”. The same applies to subsequent audio frames.

[0069] That is, in general, the silence detection queue always stores the latest N audio frames, wherein N is the number of audio frames in the silence detection queue that are not overwritten by other audio frames when the silence detection queue is full.

[0070] In one example, all the audio frames in the silence detection queue can be copied to the recognition queue in order without the silence detection queue being full, and in the case that the silence detection queue is full, the audio frames can be copied in order along the data update direction of the silence detection queue starting from the audio frame in the data space pointed to by the tail pointer (tail) to the audio frame at the position of tail-1. Figure 5 The "b4", "b5", "b6", "b7", "b8", and "b10" can be sequentially taken out in order from "b4", and the "b4", "b5", "b6", "b7", "b8", and "b10" can be sequentially stored in the recognition queue.

[0071] In this embodiment, the recognition queue is also a first input first output (FIFO) queue, and the size of the queue is fixed, that is, the newly inserted audio frame to be recognized is placed in the space pointed to by the tail pointer (tail) of the recognition queue, and the audio frame is taken from the position pointed to by the head pointer (head) of the recognition queue to the tail during the execution of the recognition task.

[0072] It should be noted that, since the tail of the recognition queue is the audio frame in the non-silence state before copying, copying the audio frame in the silence detection queue to the tail of the recognition queue can ensure that the boundary of the silence state change does not cause the head data and the tail data to be lost.

[0073] The embodiments of the present application provide an audio processing method, which comprises the following steps: first, collecting audio data, and then storing the audio data in a recording queue; then, in the case that the silence detection result of the current audio frame in the recording queue is a non-silence audio frame, the global state is a silence state, and there are a first number of continuous non-silence audio frames after the current audio frame, copying the audio frame in the silence detection queue to the recognition queue, so that the three independent tasks (i.e., the recording task, the silence detection task, and the recognition task) and the two global states (i.e., the silence state and the non-silence state) are set, and the message flow is realized through the queue between the three tasks, so that the boundary of the silence state change also does not cause the head data and the tail data to be lost.

[0074] It should be noted that each embodiment of the present application can be freely combined, the order can be changed, or each embodiment can be executed independently, and does not need to rely on or depend on a fixed execution order.

[0075] In some embodiments, the method further comprises: inserting the current audio frame into the recognition queue.

[0076] In some embodiments, the current audio frame is located after the copied audio frame in the silence detection queue in the recognition queue.

[0077] For example, if the current audio frame is "b11", and the non-silence audio frames "b4", "b5", "b6", "b7", "b8", and "b10" are stored in the silence detection queue in sequence, after copying the audio frames "b4", "b5", "b6", "b7", "b8", and "b10" to the identification queue in sequence, "b11" is inserted after "b10". In this way, when switching from the silence state to the non-silence state, the boundary of the silence state change does not cause the head data and tail data to be lost.

[0078] In some embodiments, the number of copied audio frames in the silence detection queue is greater than or equal to the first frame number.

[0079] In one example, the silence detection queue is divided into N equal-sized data spaces. In general, the latest N audio frame data is always stored in the silence detection queue. When copying the audio frames from the silence detection queue to the identification queue, the latest N audio frames stored in the silence detection queue can be copied to the identification queue, where N is the number of the latest audio frame data in the silence detection queue that is not covered by other data audio frames when the queue is full.

[0080] Therefore, in this embodiment, in order to avoid the head data and tail data being lost at the boundary of the silence state change, N can be greater than or equal to the first frame number.

[0081] In some embodiments, the method further includes:

[0082] determining that the global state is the non-silence state, and inserting the current audio frame into the identification queue.

[0083] In this embodiment, when the current audio frame is a non-silence audio frame and the global state is the non-silence state, it indicates that the current audio frame has not changed state. Therefore, the current audio frame can be directly inserted into the tail of the identification queue, and waiting for subsequent speech recognition.

[0084] Embodiments of the present application provide an audio processing method. In the case where it is detected that the current audio frame has not changed state, the current audio frame is directly inserted into the corresponding queue according to the silence detection result.

[0085] In some embodiments, after determining that there are consecutive non-silence audio frames for the first frame number, the method further includes:

[0086] updating the global state to the non-silence state.

[0087] In the embodiment, in the case that there are continuous first frame number of non-silence audio frames, it is indicated that there is a high probability of human voice, and the global state is updated to the non-silence state, so that the subsequent detected non-silence audio frames can be directly inserted into the recognition queue, avoiding the frame loss condition of subsequent speech recognition.

[0088] In some embodiments, further comprising:

[0089] In the case that the current audio frame is a non-silence audio frame, the global state is updated to the non-silence state, and the current audio frame is inserted into the non-silence detection queue.

[0090] In the embodiment, in the case that the current audio frame is a non-silence audio frame, the global state is updated to the non-silence state, and the current audio frame is inserted into the non-silence detection queue.

[0091] In some embodiments, the non-silence detection queue comprises a plurality of data spaces connected in a loop, the current audio frame is inserted into a data space pointed to by a tail pointer of the non-silence detection queue, and the tail pointer moves along a data update direction of the non-silence detection queue to a data space pointed to by a head pointer of the non-silence detection queue.

[0092] As Figures 3 to 5 , the non-silence detection queue can be regarded as a circular buffer composed of a plurality of data spaces connected in a loop. When there is no element in the non-silence detection queue, the head pointer and the tail pointer point to the same data space. When a new audio frame needs to be stored in the non-silence detection queue, the new audio frame is inserted into a data space pointed to by the tail pointer of the non-silence detection queue, and the tail pointer moves along a data update direction of the non-silence detection queue to a data space pointed to by the head pointer of the non-silence detection queue until the non-silence detection queue is full.

[0093] When the non-silence detection queue is full, the head pointer and the tail pointer again point to the same data space, so that when a new audio frame is inserted, the tail pointer continues to move along the data update direction of the non-silence detection queue to the data space pointed to by the head pointer of the non-silence detection queue.

[0094] Therefore, even if the audio frames in the non-silence detection queue are not taken out in time, when new audio frames are stored subsequently, the oldest audio frames will be covered by the latest audio frames, and the non-silence detection queue can always save a plurality of latest audio frames that are not covered by other data audio frames.

[0095] In some embodiments, after determining that the non-silence detection result of the current audio frame is a non-silence audio frame, further comprising:

[0096] determining that the global state is the non-silence state, determining that there are a second number of consecutive silence audio frames, and updating the global state to the silence state.

[0097] In the embodiment, in the case that the current audio frame is a silence audio frame and the global state is the non-silence state, it indicates that the current audio frame has a state change. It should be noted that, under the influence of the user's speaking volume and other factors, the silence detection result of part of the audio data of the human voice can also be a silence audio frame. Therefore, in the embodiment, in order to avoid the loss of frames in subsequent speech recognition, the global state is updated to the silence state only in the case that there are a second number of consecutive silence audio frames, so as to avoid invalid speech recognition behavior.

[0098] In one example, the second number of consecutive silence audio frames can include the current audio frame. For example, in some scenarios, the user can output some invalid speech data, such as "Hi…I want…um (hesitation tone)…play a song", during the process of conveying a speech instruction. In the embodiment, in order to avoid the recognition of a large amount of invalid audio data (such as "I want", "um", and the like) in the recognition task due to the above factors, it is set that the second number of consecutive non-silence audio frames includes the current audio frame.

[0099] In another example, the second number of consecutive silence audio frames can not include the current audio frame. For example, under the influence of the user's speaking habit, the user can speak slowly and the tail tone can be weak, so that there can be a plurality of consecutive silence audio frames in the detection result of the audio data during the user's speech. In the embodiment, in order to avoid the frequent updating of the global state due to the above factors and the loss of part of the audio data of the human voice, it is set that the second number of consecutive silence audio frames includes the current audio frame.

[0100] For example, for an audio frame with a frame length of 10 ms, since the duration of the interruption during the user's intermittent speech is about 100 ms in general, the second number of frames can be set to 10. Then, starting from the (N-10)th frame, when it is detected that the silence detection result of the audio frame from the (N-10)th frame to the Nth frame (that is, the current frame) is also a silence audio frame, the silence detection result of the (N+1)th frame is determined. If it is detected that the (N+1)th frame (that is, the current frame) is a non-silence audio frame, it indicates that the user's speech has not ended, otherwise, it indicates that the user's speech has ended, and the global state can be updated. In some embodiments, after it is determined that the silence detection result of the current audio frame is a silence audio frame, the method further includes:

[0101] determining that the global state is the non-silence state, determining that there are a second number of consecutive silence audio frames, and updating the global state to the silence state.

[0102] In this embodiment, when there is no continuous second number of silent audio frames, it indicates that the user's speech is not completely interrupted, for example, the user pauses for a few seconds and then starts to speak intermittently during the speech, in order to avoid affecting the processing result of the subsequent audio frames, the global state is not updated in this case.

[0103] In some embodiments, further comprising: identifying the audio frames in the identification queue in real time.

[0104] When performing a speech recognition task, the audio frames stored in the speech recognition queue are audio frames with the silent part removed, with reference to Figure 6 In this embodiment, only when a non-silent audio frame is inserted into the identification queue, the speech recognition task is called for identification, so that most of the time only the silent detection task is running, and the speech recognition task is not running, thereby saving a lot of invalid identification behavior.

[0105] With reference to Figure 7 , Figure 7 is a complete flowchart of the silent detection task in the audio processing method provided by the embodiment of the present application, comprising the following steps:

[0106] Step S1, taking an audio frame from the recording queue;

[0107] Step S2, silent detection;

[0108] Step S3, silent judgment, if the probability value of the silent detection of the current audio frame exceeds the preset threshold value, it is determined as a non-silent audio frame, and step S7 is entered, otherwise it is determined as a silent audio frame, and step S4 is entered;

[0109] Step S4, whether the previous global state is a silent state, if yes, step S6 is entered, otherwise, step S5 is entered;

[0110] Step S5, judging whether there are continuous K frames of silence before the current audio frame, if yes, the global state is updated to a silent state;

[0111] Step S6, inserting the current audio frame into the silent detection queue;

[0112] Step S7, whether the previous global state is a non-silent state, if yes, step S8 is entered, otherwise, step S9 is entered;

[0113] Step S8, inserting the current audio frame into the identification queue;

[0114] Step S9, judging whether there are continuous M frames of non-silence before the current audio frame, if yes, the global state is updated to a non-silent state, and step S10 is executed;

[0115] Step S10: Copy the audio frames in the silence detection queue to the recognition queue.

[0116] The audio processing system provided in this application is described below. The audio processing system described below can be referred to in correspondence with the audio processing method described above.

[0117] The system provided in this application is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0118] Figure 8 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application, as shown below. Figure 8 As shown, the electronic device may include: a processor 801, a communications interface 802, a memory 803, and a communication bus 804, wherein the processor 801, the communications interface 802, and the memory 803 communicate with each other through the communication bus 804. The processor 801 can call logical instructions in the memory 803 to execute an audio processing method, which includes: acquiring audio data; storing the audio data in a recording queue, wherein the recording queue is a first-in-first-out queue; retrieving the current audio frame from the recording queue; determining that the silence detection result of the current audio frame is a non-silent audio frame and the global state is silent; determining that there is a first consecutive non-silent audio frame following the current audio frame; and copying the audio frames from the silence detection queue to the end of the recognition queue, wherein the silence detection queue is a circular queue and the recognition queue is a first-in-first-out queue.

[0119] Furthermore, the logical instructions in the aforementioned memory 803 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0120] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer can execute the audio processing method provided by the above method, and the method comprises: collecting audio data; storing the audio data into a recording queue, wherein the recording queue is a first-in first-out queue; taking out a current audio frame from the recording queue; determining that the silence detection result of the current audio frame is a non-silence audio frame and the global state is a silence state; determining that there are a first number of continuous non-silence audio frames after the current audio frame; and copying the audio frames of a silence detection queue to the tail of an identification queue, wherein the silence detection queue is a circular queue, and the identification queue is a first-in first-out queue.

[0121] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer can execute the audio processing method provided by the above method, and the method comprises: collecting audio data; storing the audio data into a recording queue, wherein the recording queue is a first-in first-out queue; taking out a current audio frame from the recording queue; determining that the silence detection result of the current audio frame is a non-silence audio frame and the global state is a silence state; determining that there are a first number of continuous non-silence audio frames after the current audio frame; and copying the audio frames of a silence detection queue to the tail of an identification queue, wherein the silence detection queue is a circular queue, and the identification queue is a first-in first-out queue.

[0122] The device embodiments described above are only schematic, wherein the units illustrated as separate components may or may not be physically separate, and the components illustrated as units may or may not be physical units, i.e., may be located in one place or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0123] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary general hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0124] It should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same. Although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some of the technical features can be replaced by equivalent features. Such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An audio processing method, characterized by, The method comprises: acquiring audio data; storing the audio data into a recording queue, wherein the recording queue is a first-in-first-out queue; taking out a current audio frame from the recording queue; when it is determined that a silence detection result of the current audio frame is a non-silence audio frame and a global state is a silence state; determining that there are a first number of continuous non-silence audio frames; copying audio frames of a silence detection queue to a recognition queue, wherein the silence detection queue is a circular queue and the recognition queue is a first-in-first-out queue; wherein the number of the copied audio frames in the silence detection queue is greater than or equal to the first number of frames, and the audio frames in the recognition queue are recognized in real time.

2. The method of claim 1, wherein, The method further comprises: inserting the current audio frame into the recognition queue, wherein the current audio frame is located after the copied audio frames of the silence detection queue in the recognition queue.

3. The method of claim 1, wherein, The method further comprises: when it is determined that the global state of the current audio frame is a non-silence state, inserting the current audio frame into the recognition queue.

4. The method of claim 1, wherein, The method further comprises: updating the global state to a non-silence state.

5. The method of claim 1, wherein, The method further comprises: when it is determined that the silence detection result of the current audio frame is a silence audio frame, inserting the current audio frame into the silence detection queue.

6. The method of claim 5, wherein, The silence detection queue comprises a plurality of data spaces connected in a loop, and the current audio frame is inserted into a data space pointed to by a tail pointer of the silence detection queue; the tail pointer moves to a data space pointed to by a head pointer of the silence detection queue along a data updating direction of the silence detection queue.

7. The method of claim 5, wherein, The method further comprises: when it is determined that the global state of the current audio frame is a non-silence state, determining that there are a second number of continuous silence audio frames, and updating the global state to a silence state.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the audio processing method according to any one of claims 1 to 7 when executing the computer program.

Images