Voice processing method, processor, voice processing device and electronic equipment

Abstract

The embodiment of the invention discloses a voice processing method, a processor, a voice processing device and electronic equipment, relates to the technical field of communication, and solves the problems that when a user sends a command to the electronic equipment through voice, if the user changes the intention, the electronic equipment is slow in response speed and not in time in response. According to the specific scheme, the voice processing method is provided and applied to a processor, and the method comprises the following steps: firstly, obtaining first voice data; and then, determining a feature sequence corresponding to the first voice data, and skipping execution of a first instruction corresponding to the first voice data when the feature sequence comprises a preset feature and the preset feature is located at the tail of the feature sequence.

Description

Technical Field

[0001] This application relates to the field of communication technologies, and in particular, to a voice processing method, a processor, a voice processing device, and an electronic device. Background Art

[0002] As a new generation of user interaction mode following keyboard interaction, mouse interaction, and touch screen interaction, voice interaction is gradually favored by the majority of users with its convenient and fast characteristics and is widely used in various electronic devices. Currently, when a user conducts voice interaction with an electronic device, the user sends a command to the electronic device through voice, and the electronic device converts the voice into an executable instruction and performs corresponding operations according to the instruction. After the electronic device completes the corresponding operations, the user can send the next command to the electronic device through voice again.

[0003] However, when a user sends a command to an electronic device through voice, if the user changes the intention, the electronic device has problems of slow response speed and untimely response. Summary of the Invention

[0004] Embodiments of this application provide a voice processing method, a processor, a voice processing device, and an electronic device, which solve the problems that when a user sends a command to an electronic device through voice, if the user changes the intention, the electronic device has slow response speed and untimely response.

[0005] To achieve the above object, the embodiments of this application adopt the following technical solutions:

[0006] In a first aspect of the embodiments of this application, a voice processing method is provided, which is applied to a processor. The method includes: First, obtain first voice data. Then, determine a feature sequence corresponding to the first voice data. When the feature sequence includes a preset feature and the preset feature is located at the tail of the feature sequence, skip the execution of a first instruction corresponding to the first voice data.

[0007] In a possible embodiment, the above preset feature is used to indicate that the user's intention has changed.

[0008] Optionally, the preset feature may be a feature corresponding to a transition word such as "wait", "wait a moment", or "oh no" that indicates a transition meaning. The embodiments of this application do not limit this.

[0009] Optionally, the above skipping the execution of the first instruction corresponding to the first voice data may be equivalent to not executing, abandoning the execution, or ignoring the execution of the first instruction corresponding to the first voice data. The embodiments of this application do not limit this.

[0010] Based on this solution, first, the processor obtains the first voice data. Then, the processor determines the feature sequence corresponding to the first voice data. When the feature sequence includes a preset feature and the preset feature is located at the end of the feature sequence, the execution of the first instruction corresponding to the first voice data is skipped. From the user's perspective, when the user's intention changes, the processor does not execute the first instruction before the intention change, and the user will feel that the processor has a fast reaction speed and responds in a timely manner. Thus, when this processor is applied to a voice processing device and the voice processing device is applied to an electronic device, the user will feel that the electronic device has a fast reaction speed and responds in a timely manner.

[0011] Combined with the first aspect, in a possible implementation, when the first voice data includes the turning voice data corresponding to a turning word and the turning voice data is located at the end of the first voice data, the feature sequence includes the preset feature corresponding to the turning word and the preset feature is located at the end of the feature sequence.

[0012] Combined with the first aspect, in a possible implementation, after obtaining the first voice data, the method further includes: obtaining the second voice data. When it is determined that the first voice data and the second voice data are complete voice data, the second instruction corresponding to the second voice data is executed.

[0013] Based on this solution, the processor obtains the second voice data. When it is determined that the first voice data and the second voice data are complete voice data, the second instruction corresponding to the second voice data is executed. From the user's perspective, the processor does not execute the first instruction before the user's intention change, but executes the second instruction after the user's intention change, and the user will feel that the processor has a fast reaction speed and responds in a timely manner. Secondly, when it is determined that the first voice data and the second voice data are complete voice data, the processor executes the second instruction, which can more accurately judge the user's intention and improve the user's voice interaction experience.

[0014] Combined with the first aspect, in a possible implementation, the above-mentioned execution of the second instruction corresponding to the second voice data includes: after obtaining the second voice data and when no voice data is obtained within a preset duration, the second instruction is executed.

[0015] Based on this solution, after the processor obtains the second voice data and when no voice data is obtained within a preset duration, the processor can determine that the user has stopped speaking, and the processor can more accurately confirm the user's intention. At this time, when the processor executes the second instruction, the voice interaction experience between the user and the voice processing device 100 can be further improved.

[0016] In combination with the first aspect, in a possible implementation manner, the above-mentioned execution of the second instruction corresponding to the second voice data includes: obtaining the first voiceprint feature corresponding to the voice data, and when the similarity between the first voiceprint feature and the second voiceprint feature is greater than or equal to a preset value, executing the second instruction, where the second voiceprint feature is the voiceprint feature of the target user.

[0017] Based on this solution, when the similarity between the first voiceprint feature and the second voiceprint feature is greater than or equal to a preset value, the processor executes the second instruction. When this processor is applied to a voice processing device, the security of the interaction between the user and the voice processing device can be improved.

[0018] In combination with the first aspect, in a possible implementation manner, the above-mentioned determination of the feature sequence corresponding to the first voice data includes: determining the feature sequence corresponding to the first voice data through Mel-Frequency Cepstral Coefficients (MFCC).

[0019] In the second aspect of the embodiments of the present application, a processor is provided. The processor includes an interface circuit and a computing circuit that are coupled to each other. Among them, the interface circuit is used to obtain the first voice data. The computing circuit is used to determine the feature sequence corresponding to the first voice data, and when the feature sequence includes a preset feature and the preset feature is located at the end of the feature sequence, skip the execution of the first instruction corresponding to the first voice data.

[0020] In combination with the second aspect, in a possible implementation manner, when the first voice data includes the turning voice data corresponding to a turning word and the turning voice data is located at the end of the first voice data, the feature sequence includes the preset feature corresponding to the turning word, and the preset feature is located at the end of the feature sequence.

[0021] In combination with the second aspect, in a possible implementation manner, after the interface circuit obtains the first voice data, the interface circuit is further used to obtain the second voice data. The computing circuit is further used to execute the second instruction corresponding to the second voice data when it is determined that the first voice data and the second voice data are complete voice data.

[0022] In combination with the second aspect, in a possible implementation manner, the computing circuit is specifically used to execute the second instruction after obtaining the second voice data and when no voice data is obtained within a preset duration.

[0023] In combination with the second aspect, in a possible implementation manner, the computing circuit is specifically used to obtain the first voiceprint feature corresponding to the voice data, and when the similarity between the first voiceprint feature and the second voiceprint feature is greater than or equal to a preset value, execute the second instruction, where the second voiceprint feature is the voiceprint feature of the target user.

[0024] In combination with the second aspect, in a possible implementation manner, the computing circuit is specifically used to determine the feature sequence corresponding to the first voice data through Mel-Frequency Cepstral Coefficients (MFCC).

[0025] In the third aspect of the embodiments of the present application, a voice processing device is provided. The voice processing device includes a sound collector and a processor that are coupled to each other. Among them, the sound collector is used to collect voice data, and the processor is the processor described in the second aspect or any possible implementation manner of the second aspect above.

[0026] In combination with the third aspect, in a possible implementation manner, the voice processing device further includes a voice enhancement circuit coupled between the sound collector and the processor. The voice enhancement circuit is used to perform voice enhancement processing on the voice data, and the voice enhancement processing includes at least one of acoustic echo cancellation (AEC), direction of arrival (DOA) source localization, blind source separation (BSS), dereverberation, or noise reduction.

[0027] In combination with the third aspect, in a possible implementation manner, the processor is further used to perform noise reduction processing on the voice data after voice enhancement processing when the signal-to-noise ratio of the voice data after voice enhancement processing is less than a preset signal-to-noise ratio.

[0028] In the fourth aspect of the embodiments of the present application, an electronic device is provided. The electronic device includes a main board and a voice processing device fixed to the main board. The voice processing device is the voice processing device described in the third aspect or any possible implementation manner of the third aspect above.

[0029] In the fifth aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. When the computer program runs on a processor, the processor is caused to execute the voice processing method described in the first aspect or any possible implementation manner of the first aspect above.

[0030] In the sixth aspect of the embodiments of the present application, a computer program product is provided. When a processor executes the computer program product, the processor is caused to execute the voice processing method described in the first aspect or any possible implementation manner of the first aspect above.

[0031] The descriptions of the second aspect to the sixth aspect in the present application can refer to the detailed description of the first aspect; and, the beneficial effects described in the second aspect to the sixth aspect can refer to the analysis of the beneficial effects of the first aspect, which will not be elaborated here. Description of the Drawings

[0032] Figure 1 It is a schematic structural diagram of a voice processing device provided by an embodiment of the present application;

[0033] Figure 2 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0034] Figure 3Schematic flowchart of a voice processing method provided by an embodiment of the present application;

[0035] Figure 4 Schematic flowchart of another voice processing method provided by an embodiment of the present application;

[0036] Figure 5 Schematic diagram of text recognition of voice data provided by an embodiment of the present application;

[0037] Figure 6 Schematic structural diagram of a processor provided by an embodiment of the present application;

[0038] Figure 7 Schematic structural diagram of another voice processing device provided by an embodiment of the present application;

[0039] Figure 8 Schematic structural diagram of yet another voice processing device provided by an embodiment of the present application. Detailed implementation manners

[0040] The fabrication and use of the embodiments will be discussed in detail below. However, it should be understood that many applicable inventive concepts provided by the present application can be implemented in a variety of specific environments. The specific embodiments discussed are merely used to illustrate the specific ways of implementing and using this specification and this technology, and do not limit the scope of the present application.

[0041] Unless otherwise defined, all scientific and technical terms used herein have the same meaning as commonly understood by those of ordinary skill in the art.

[0042] Each circuit or other component may be described or referred to as "configured to" perform one or more tasks. In this case, "configured to" is used to imply a structure by indicating that the circuit / component includes a structure (such as circuitry) that performs one or more tasks during operation. Thus, even when the specified circuit / component is currently inoperable (e.g., not turned on), the circuit / component can still be referred to as configured to perform the task. Circuits / components used in conjunction with the phrase "configured to" include hardware, such as circuitry that performs the operations, etc.

[0043] Next, the technical solutions in the embodiments of the present application will be described in conjunction with the accompanying drawings in the embodiments of the present application. In the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item)" or similar expressions thereof refer to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple. Additionally, in the embodiments of the present application, words such as "first" and "second" do not limit the quantity and order.

[0044] In the present application, words such as "exemplary" or "for example" are used to give examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0045] Before introducing the embodiments of the present application, the technical terms and background technologies related to the present application will be introduced first.

[0046] Mel-scale frequency cepstral coefficients (MFCC): It is a technology widely used in the field of speech processing for extracting the features of speech data.

[0047] Voiceprint: It is the sound wave spectrum carrying speech information displayed by electroacoustic instruments.

[0048] Voiceprint feature: It refers to the unique features that distinguish each person's voice from others.

[0049] Currently, when a user interacts with an electronic device through voice, the user sends a command to the electronic device through voice. The electronic device converts the voice into an executable instruction and performs corresponding operations according to the instruction. After the electronic device finishes performing the corresponding operations, the user can send the next command to the electronic device through voice again.

[0050] However, when the user sends a command to the electronic device through voice, if the user changes the intention, the electronic device has problems of slow response speed and untimely response.

[0051] For example, after a user sends a first command to an electronic device via voice, if the user changes their intention and sends a second command to the electronic device via voice again, and the user hopes that the electronic device executes the second instruction corresponding to the second command instead of the first instruction corresponding to the first command, the electronic device will execute the first instruction corresponding to the first command instead of the second instruction corresponding to the second command. From the user's perspective, since the electronic device does not execute the second instruction corresponding to the second command in a timely manner, the user will feel that the electronic device has problems with slow response speed and untimely response.

[0052] Based on this, an embodiment of the present application provides a voice processing method applied to a processor. This method obtains the voice data of the user and determines the feature sequence corresponding to the voice data. When the feature sequence includes a preset feature and the preset feature is located at the end of the feature sequence, the execution of the instruction corresponding to the voice data is skipped, so that when the user changes their intention, a faster response speed can be achieved and a timely response can be made.

[0053] When the voice processing method provided by the embodiment of the present application is applied to a processor, the processor can be applied to a voice processing device. As Figure 1 shown, it is a schematic structural diagram of a voice processing device 100 provided by an embodiment of the present application. The voice processing device 100 includes a processor 110 and a sound collector 120 that are coupled to each other.

[0054] The voice processing device 100 provided by the embodiment of the present application can be applied to an electronic device with voice processing functions. Exemplarily, the types of the electronic device may include but are not limited to: mobile phone, tablet computer, laptop computer, handheld computer, mobile internet device (MID), camera, wearable device (such as smart watch, smart bracelet, pedometer, etc.), audio device, audio and video player, set-top box, game console, printer, vehicle-mounted device (such as devices on vehicles such as cars, bicycles, electric vehicles, airplanes, ships, trains, and high-speed rails), virtual reality (VR) device, augmented reality (AR) device, smart home device (such as refrigerator, TV, air conditioner, electricity meter, etc.), smart robot, and wireless terminals in self-driving, etc.

[0055] For example, as Figure 2 shown, it is a schematic structural diagram of an electronic device 200 provided by an embodiment of the present application. The electronic device 200 includes a main board 210 and a voice processing device 100 fixed to the main board 210.

[0056] As Figure 3As shown in the figure, it is a schematic flowchart of a voice processing method provided by an embodiment of the present application. This method can be applied to the processor 110 in the above-mentioned voice processing device 100. In the following embodiments of the present application, taking this method as an example of being applied to the processor 110 in the above-mentioned voice processing device 100, an exemplary description is given. This method includes steps S301 - S302.

[0057] S301. The processor 110 acquires first voice data.

[0058] S302. The processor 110 determines a feature sequence corresponding to the first voice data. When the feature sequence includes a preset feature and the preset feature is located at the end of the feature sequence, the execution of the first instruction corresponding to the first voice data is skipped.

[0059] In a possible embodiment, the above-mentioned processor 110 is specifically used to determine the feature sequence corresponding to the first voice data through Mel - Frequency Cepstral Coefficients (MFCC). The specific process can refer to the prior art, and the embodiments of the present application will not elaborate on this here.

[0060] The above - mentioned preset feature is used to indicate a change in the user's intention.

[0061] Optionally, the preset feature can be a feature corresponding to a transition word indicating a transition meaning, such as "wait", "wait a minute", or "oh no", etc. The embodiments of the present application do not limit this.

[0062] For example, taking the example that the user says "Please open the music software, wait a minute" to the voice processing device 100. The sound collector 120 in the voice processing device 100 can collect the first voice data corresponding to "Please open the music software, wait a minute". The processor 110 can acquire this first voice data and determine the feature sequence corresponding to this first voice data. Since the feature sequence includes the preset feature corresponding to the transition word "wait a minute" and this preset feature is located at the end of the feature sequence, the processor 110 skips the execution of the first instruction corresponding to this first voice data and does not open the music software. From the user's perspective, when the user's intention changes, the processor 110 skips the execution of the first instruction before the intention change, and the user will feel that the voice processing device 100 has a fast response speed and responds in a timely manner. When this voice processing device 100 is applied to the electronic device 200, the user will feel that the electronic device 200 has a fast response speed and responds in a timely manner.

[0063] Optionally, the above - mentioned skipping the execution of the first instruction corresponding to the first voice data can be equivalent to not executing, abandoning the execution, or ignoring the execution of the first instruction corresponding to the first voice data. The embodiments of the present application do not limit this.

[0064] In a possible embodiment, when the first voice data includes turning voice data corresponding to a turning word, and the turning voice data is located at the tail of the first voice data, the feature sequence corresponding to the first voice data includes a preset feature corresponding to the turning word, and the preset feature is located at the tail of the feature sequence.

[0065] In a possible embodiment, the above-mentioned processor 110 may be a digital signal processing (DSP), or may be a microcontroller unit (MCU). The embodiments of the present application do not limit this.

[0066] In the voice processing method provided by the embodiments of the present application, first, the processor 110 obtains the first voice data. Then, the processor 110 determines the feature sequence corresponding to the first voice data. When the feature sequence includes a preset feature and the preset feature is located at the tail of the feature sequence, the execution of the first instruction corresponding to the first voice data is skipped. From the user's perspective, when the intention of the user changes, the processor 110 does not execute the first instruction before the intention change, and the user will feel that the processor 110 has a fast reaction speed and is responsive. Thus, when the processor 110 is applied to the voice processing device 100 and the voice processing device 100 is applied to the electronic device 200, the user will feel that the electronic device 200 has a fast reaction speed and is responsive.

[0067] As Figure 4 shown, in a possible embodiment, after the processor 110 obtains the first voice data, the method further includes steps S303 - S304. When executing steps S303 - S304 and step S302, the execution order may not be distinguished, for example, they may be executed simultaneously.

[0068] S303. The processor 110 obtains the second voice data.

[0069] S304. When the processor 110 determines that the first voice data and the second voice data are complete voice data, execute the second instruction corresponding to the second voice data.

[0070] In a possible embodiment, the processor 110 is specifically configured to determine that the first voice data and the second voice data are complete voice data when identifying the text corresponding to the first voice data and the second voice data through an acoustic language model and determining that the text is a complete text through a text integrity recognition model. The specific process of the processor 110 identifying the text through the acoustic language model and the specific process of determining that the text is a complete text through the integrity recognition model may refer to the prior art, and the embodiments of the present application will not elaborate herein.

[0071] Optionally, the acoustic language model and the text integrity recognition model can be the same model, or they can be different models. The embodiments of the present application do not limit this. In the following embodiments of the present application, an example in which the acoustic language model and the text integrity recognition model are the same model is used for exemplary illustration.

[0072] Optionally, the above text integrity recognition model may include at least one of a deep neural network (DNN), a recurrent neural network (RNN), a hidden Markov model (HMM), or a Transformer model (encoder-decoder Encoder-Decoder architecture), and may also include existing or future other algorithms or models that can perform phoneme information and text feature extraction and classification. The embodiments of the present application do not limit this.

[0073] For example, taking the processor 110 as being applied to the voice processing device 100, the voice processing device 100 as being applied to the electronic device 200, the electronic device 200 also including a speaker, the text integrity recognition model being a deep neural network, and the user saying "Please open the music software, wait a moment" to the electronic device 200 and then saying "Please open the picture software" to the electronic device 200 as an example.

[0074] As Figure 5 shown, first, the sound collector 120 can collect the first voice data corresponding to "Please open the music software, wait a moment". The processor 110 can obtain the first voice data and determine the feature sequence corresponding to the first voice data. Since the feature sequence includes a preset feature corresponding to the transition word "wait a moment" and the preset feature is at the end of the feature sequence, the processor 110 can skip executing the first instruction corresponding to the first voice data and not open the music software. The electronic device 200 can reply "Um" to the user through the speaker. The sound collector 120 can continue to collect the second voice data corresponding to "Please open the picture software".

[0075] Then, the processor 110 can obtain the second voice data and, through the deep neural network, recognize the text corresponding to the first voice data and the second voice data, which is "Please open the music software, wait a moment, please open the picture software".

[0076] Finally, the processor 110 can, through the deep neural network model, based on the interaction between the user and the electronic device 200, when determining that the text is a complete text, execute the second instruction corresponding to the second voice data and open the picture software. The electronic device 200 can reply to the user through the speaker with "Okay, the picture software has been opened for you".

[0077] It can be understood that by adopting the voice processing method provided in the embodiments of the present application, after collecting the first voice data of the user, if it is recognized that the user's intention has changed, the first instruction corresponding to the first voice data will not be executed. Instead, the second voice data of the user will continue to be collected. When the texts corresponding to the first voice data and the second voice data are recognized and it is determined by the text integrity recognition model that the text is a complete text, the second instruction corresponding to the second voice data will be executed. From the user's perspective, the electronic device 200 does not execute the first instruction before the user's intention changes, but executes the second instruction after the user's intention changes. The user will feel that the voice processing device 100 has a fast response speed and is responsive. Secondly, when it is determined that the first voice data and the second voice data are complete voice data, executing the second instruction can more accurately judge the user's intention and improve the user's voice interaction experience.

[0078] In a possible embodiment, when the processor 110 is applied to the voice processing device 100 and the voice processing device 100 is applied to the electronic device 200, the electronic device 200 may further include a memory, and the above-mentioned text integrity model may be stored in the memory. Thus, when the electronic device 200 is not connected to the network, the voice processing device 100 can run offline and still support simple voice interaction between the user and the electronic device 200. Moreover, since the text integrity model is stored in the memory and there is no need to transmit data through the network, the response speed of the voice processing device 100 can be improved and the conversation delay between the user and the electronic device 200 can be reduced. Alternatively, the text integrity model may be stored in a server in the cloud. The text integrity model is not limited by computing resources and storage space and can have more powerful functions. When the electronic device 200 is connected to the network, more complex voice interaction can be realized through the text integrity model stored in the cloud, further improving the user's voice interaction satisfaction. The embodiments of the present application do not limit whether the text integrity model is specifically stored in the memory or in a server in the cloud. For example, a relatively small-scale text integrity model may be stored in the memory, and a relatively large-scale text integrity model may be stored in a server in the cloud.

[0079] In a possible embodiment, the above-mentioned processor 110 executes the second instruction corresponding to the second voice data, including: after the processor 110 obtains the second voice data and no voice data is obtained within a preset duration, the second instruction is executed. The embodiments of the present application do not limit the specific duration of the preset duration.

[0080] In a possible embodiment, when the processor 110 is applied to the above-mentioned voice processing device 100, the sound collector 120 in the voice processing device 100 can also be used to detect silence. When the sound collector 310 detects silence within a preset duration, the above-mentioned processor 110 will not obtain voice data within the preset duration. Among them, the sound collector 310 detecting silence within the preset duration includes: the duration of the sound collector 120 detecting silence being equal to the preset duration, or the duration of the sound collector 120 detecting silence being greater than the preset duration. Thus, when the sound collector 310 detects silence within the preset duration and the processor 110 does not obtain voice data within the preset duration, the processor 110 can determine that the user has stopped speaking, can more accurately confirm the user's intention, and can further improve the voice interaction experience between the user and the voice processing device 100.

[0081] For example, as Figure 5 shown, taking the preset duration as 1 second as an example, after the sound collector 120 finishes collecting the second voice data, when the duration of the sound collector 120 detecting silence is equal to 1 second, it can be understood that the sound collector 120 does not collect voice data within the preset duration of 1 second, and the processor 110 does not obtain voice data within the preset duration of 1 second. The processor 110 can determine that the user has stopped speaking, execute the second instruction corresponding to the second voice data, open the picture software, and the electronic device 200 can reply to the user through the speaker "Okay, the picture software has been opened for you".

[0082] In a possible embodiment, the above-mentioned processor 110 executing the second instruction corresponding to the second voice data includes: the processor 110 obtaining the first voiceprint feature corresponding to the voice data, and when the similarity between the first voiceprint feature and the second voiceprint feature is greater than or equal to a preset value, executing the second instruction, where the second voiceprint feature is the voiceprint feature of the target user, and the specific value of this preset value is not limited in the embodiments of the present application. Thus, when the processor 110 is applied to the voice processing device 100, the security during the interaction between the user and the voice processing device 100 can be improved.

[0083] Optionally, when the processor 110 is applied to the above-mentioned voice processing device 100, the above-mentioned first voice data and second voice data can be data generated by the same user collected by the sound collector 120, or can be data generated by different users collected by the sound collector 120. The embodiments of the present application do not limit this, and the embodiments of the present application take the first voice data and the second voice data as data generated by the same user collected by the sound collector 120 as an example for exemplary illustration.

[0084] For example, the processor 110 can determine the first voiceprint feature corresponding to the first voice data and the second voice data through voiceprint recognition (VPR). When the similarity between the first voiceprint feature and the second voiceprint feature is equal to a preset value, the processor 110 executes the second instruction.

[0085] In the voice processing method provided by the embodiments of the present application, after the processor 110 obtains the first voice data, the processor 110 obtains the second voice data. When the processor 110 determines that the first voice data and the second voice data are complete voice data, the second instruction corresponding to the second voice data is executed, which can more accurately judge the user's intention and improve the user's voice interaction experience.

[0086] Based on this, as Figure 6 shown, the embodiments of the present application further provide a processor 110, which includes an interface circuit 111 and a computing circuit 112 that are coupled to each other.

[0087] Among them, the interface circuit 111 is used to obtain the first voice data. The computing circuit 112 is used to determine the feature sequence corresponding to the first voice data. When the feature sequence includes a preset feature and the preset feature is located at the end of the feature sequence, the execution of the first instruction corresponding to the first voice data is skipped.

[0088] In a possible embodiment, the computing circuit 112 is specifically used to determine the feature sequence corresponding to the first voice data through Mel Frequency Cepstral Coefficients (MFCC). The specific process can refer to the prior art, and the embodiments of the present application will not elaborate here.

[0089] The above preset feature is used to indicate a change in the user's intention.

[0090] Optionally, the preset feature can be a feature corresponding to a transition word such as "wait", "wait a minute", or "oh no" that indicates a transition meaning. The embodiments of the present application do not limit this.

[0091] Optionally, the above skipping the execution of the first instruction corresponding to the first voice data can be equivalent to not executing, abandoning the execution, or ignoring the execution of the first instruction corresponding to the first voice data. The embodiments of the present application do not limit this.

[0092] In a possible embodiment, when the first voice data includes the transition voice data corresponding to the transition word and the transition voice data is located at the end of the first voice data, the feature sequence includes the preset feature corresponding to the transition word and the preset feature is located at the end of the feature sequence.

[0093] The processor 110 provided in the embodiment of the present application obtains the first voice data through the interface circuit 111, determines the feature sequence corresponding to the first voice data through the calculation circuit 112, and skips the execution of the first instruction corresponding to the first voice data when the feature sequence includes a preset feature and the preset feature is located at the end of the feature sequence. From the user's perspective, when the user's intention changes, the processor 110 does not execute the first instruction before the intention change, and the user will feel that the processor 110 has a fast reaction speed and is responsive. Thus, when the processor 110 is applied to the voice processing device 100 and the voice processing device 100 is applied to the electronic device 200, the user will feel that the electronic device 200 has a fast reaction speed and is responsive.

[0094] In a possible embodiment, after obtaining the first voice data, the interface circuit 111 is further configured to obtain the second voice data. The calculation circuit 112 is further configured to execute the second instruction corresponding to the second voice data when it is determined that the first voice data and the second voice data are complete voice data.

[0095] In a possible embodiment, the calculation circuit 112 is specifically configured to determine that the first voice data and the second voice data are complete voice data when identifying the text corresponding to the first voice data and the second voice data through an acoustic language model and determining that the text is a complete text through a text integrity recognition model. The specific process of the calculation circuit 112 identifying the text through the acoustic language model and the specific process of determining that the text is a complete text through the integrity recognition model can refer to the prior art, and the embodiments of the present application will not elaborate herein.

[0096] Optionally, the acoustic language model and the text integrity recognition model may be the same model, or may be different models. The embodiments of the present application do not limit this, and the following embodiments of the present application will take the acoustic language model and the text integrity recognition model as the same model as an example for illustrative purposes.

[0097] Optionally, the above text integrity recognition model may include at least one of a deep neural network, a recurrent neural network, a hidden Markov model, or a Transformer model, and may also include existing or future other algorithms or models that can perform phoneme information and text feature extraction and classification. The embodiments of the present application do not limit this.

[0098] In a possible embodiment, the computing circuit 112 is specifically configured to execute a second instruction after obtaining the second voice data and when no voice data is obtained within a preset duration. The specific duration of the preset duration in the embodiments of the present application is not limited. Thus, when the processor 110 is applied to the voice processing device 100, it can be determined whether the user stops speaking, the user's intention can be more accurately confirmed, and the voice interaction experience between the user and the voice processing device 100 can be further improved.

[0099] In a possible embodiment, the computing circuit 112 is specifically configured to obtain a first voiceprint feature corresponding to voice data, and execute a second instruction when the similarity between the first voiceprint feature and a second voiceprint feature is greater than or equal to a preset value. The second voiceprint feature is the voiceprint feature of the target user. The specific value of the preset value in the embodiments of the present application is not limited. Thus, when the processor 110 is applied to the voice processing device 100, the security during the interaction between the user and the voice processing device 100 can be improved.

[0100] Optionally, when the processor 110 is applied to the above-mentioned voice processing device 100, the above-mentioned first voice data and second voice data may be data generated by the sound collector 120 collecting the same user, or may be data generated by the sound collector 120 collecting different users. The embodiments of the present application do not limit this.

[0101] In the processor 110 provided in the embodiments of the present application, after the interface circuit 111 obtains the first voice data, the interface circuit 111 is further configured to obtain the second voice data. The computing circuit 112 is further configured to execute a second instruction corresponding to the second voice data when it is determined that the first voice data and the second voice data are complete voice data, so that the user's intention can be more accurately judged and the user's voice interaction experience can be improved.

[0102] As Figure 1 shown, the embodiments of the present application further provide a voice processing device 100, which includes a processor 110 and a sound collector 120 that are coupled to each other.

[0103] Among them, the sound collector 120 is configured to collect voice data. The processor 110 is configured to obtain the voice data collected by the sound collector 120. The structure of the processor 110 is the structure of the above-mentioned Figure 6 processor 110. The related description of the processor 110 can refer to the above-mentioned Figure 3 or Figure 4 described voice processing method, and Figure 6 the related description of the above-mentioned processor 110.

[0104] As Figure 7As shown, in a possible embodiment, the sound collector 120 includes at least one microphone 121 and an Always on Microphone activity detection (Always On MAD) circuit 122 coupled to the at least one microphone 121. In the embodiments of the present application, the specific number of microphones included in the at least one microphone 121 is not limited. In the following embodiments, an example is given where the at least one microphone 121 includes 4 microphones for illustrative purposes.

[0105] Among them, the at least one microphone 121 is used to collect the user's voice signal and convert the voice signal into an analog signal. The Always on Microphone activity detection circuit 122 is used to convert the analog signal into a digital signal and implement a voice wake-up function and / or a mute detection function based on the digital signal. The digital signal can also be referred to as voice data. For example, it can be the above-mentioned first voice data.

[0106] For example, as Figure 7 shown, taking the at least one microphone 121 including 4 microphones as an example, the Always on Microphone activity detection circuit 122 may include 4 analog-to-digital converters (ADCs) 1221, an anti-blocking microphone 1222, and a microphone activity detection circuit MAD 1223. One end of the 4 ADCs 1221 is respectively coupled to the 4 microphones, and the other end of the 4 ADCs 1221 is respectively coupled to one end of the anti-blocking microphone 1222. The other end of the anti-blocking microphone 1222 is coupled to the MAD 1223. The 4 ADCs 1221 are used to convert the analog signal into a digital signal. The anti-blocking microphone 1222 is used to detect whether the 4 microphones are blocked and select the unblocked microphone to collect the user's voice signal. The MAD 1223 is used to determine whether to perform voice wake-up or perform mute detection based on the digital signal output by the anti-blocking microphone 1222 and a preset threshold.

[0107] The voice processing device 100 provided in the embodiments of the present application can collect the user's first voice data through the sound collector 120, obtain the feature sequence corresponding to the first voice data through the processor 110, and skip the execution of the first instruction corresponding to the first voice data when the feature sequence includes a preset feature and the preset feature is located at the end of the feature sequence. From the user's perspective, when the user's intention changes, the voice processing device 100 does not execute the first instruction before the intention change, and the user will feel that the voice processing device 100 has a fast response speed and is responsive.

[0108] In a possible embodiment, as Figure 8As shown, the voice processing device 100 further includes a memory 130 coupled to the processor 110. The memory 130 is used to store the second voiceprint feature of the target user described above.

[0109] In the voice processing device 100 provided by the embodiment of the present application, the sound collector 120 collects the second voice data of the user. When the processor 110 determines that the first voice data and the second voice data are complete voice data, the processor 110 executes the second instruction corresponding to the second voice data. From the user's perspective, the voice processing device 100 does not execute the first instruction before the user's intention changes, but executes the second instruction after the user's intention changes. The user will feel that the voice processing device 100 has a fast response speed and responds in a timely manner. Moreover, when it is determined that the first voice data and the second voice data are complete voice data, the processor 110 executes the second instruction, which can more accurately judge the user's intention and improve the user's voice interaction experience.

[0110] In a possible embodiment, as Figure 8 shown, the voice processing device 100 may further include a voice enhancement circuit 140 coupled between the sound collector 120 and the processor 110. The voice enhancement circuit 140 is used to perform voice enhancement processing on the voice data. The voice enhancement processing includes at least one of acoustic echo canceller (AEC), direction of arrival (DOA), blind source separation (BSS), reverberation removal, or noise reduction.

[0111] Specifically, the voice enhancement circuit 140 may include an echo cancellation circuit 141, a beamforming circuit 142, a reverberation removal circuit 143, and a noise reduction circuit 144 that are sequentially coupled. It may also include a sound source localization circuit 145 and a blind source separation circuit 146. One end of the sound source localization circuit 145 is coupled to the echo cancellation circuit 141, and the other end of the sound source localization circuit 145 is coupled to the beamforming circuit 142. One end of the blind source separation circuit 146 is coupled to the echo cancellation circuit 141, and the other end of the blind source separation circuit 146 is coupled to the noise reduction circuit 144.

[0112] Among them, taking the voice processing device 100 as an example applied to the above-mentioned electronic device 200, where the electronic device 200 includes a speaker, the echo cancellation circuit 141 is used to cancel the sound of the speaker played externally collected by the microphone. The sound source localization circuit 145 is used to determine the position and direction of the sound source in space, and the beamforming circuit 142 is used to receive the voice data according to the position and direction. Specifically, as Figure 8As shown, taking the example that at least one microphone 121 includes 4 microphones, the sound source localization circuit 145 can determine the position and direction of the sound source in space, and send the parameters ω1-ω4 corresponding to each microphone to the beamforming circuit 142. The beamforming circuit 142 can adjust the voice data received by each microphone according to the parameters ω1-ω4, and combine the adjusted voice data to enhance the voice data in the direction and position of the sound source. The dereverberation circuit 143 is used to eliminate the reverberation effect in the voice data. The blind source separation circuit 146 is used to distinguish different users when multiple users perform voice interaction with the voice processing device 100.

[0113] In a possible embodiment, as Figure 8 shown, the voice enhancement circuit 140 and the always-on microphone activity detection circuit 122 in the sound collector 120 can be one circuit, and this circuit can be called an audio codec (AUDIOCODEC) 150. The audio codec 150 may also include other circuits, and the embodiments of the present application do not limit this.

[0114] The voice processing device 100 provided by the embodiments of the present application performs voice enhancement processing on voice data through the voice enhancement circuit 140, thereby improving the quality of the voice data, improving the purity of the voice data, enhancing the accuracy of voice recognition, and thus improving the user's voice interaction satisfaction.

[0115] In a possible embodiment, the processor 110 is further configured to perform noise reduction processing on the voice data after voice enhancement processing when the signal-to-noise ratio of the voice data after voice enhancement processing is less than a preset signal-to-noise ratio. The embodiments of the present application do not limit the specific value of the preset signal-to-noise ratio.

[0116] In a possible embodiment, the processor 110 can perform AI noise reduction processing on the voice data after voice enhancement processing through an artificial intelligence (AI) noise reduction model.

[0117] Optionally, the above AI noise reduction model may include at least one of a deep neural network, a convolutional neural network (CNN), a gated recurrent unit (GRU), or a Counterfactual Recurrent Network (CRN), and may also include existing or future other algorithms or models that can perform AI noise reduction. The embodiments of the present application do not limit this.

[0118] For the voice processing device 100 provided in the embodiment of the present application, when the signal-to-noise ratio of the voice data after voice enhancement processing is less than the preset signal-to-noise ratio, the voice data after voice enhancement processing is denoised again, which can further improve the quality of the voice data, improve the purity of the voice data, and enhance the accuracy of voice recognition, thereby further improving the user's voice interaction satisfaction.

[0119] Based on this, the embodiment of the present application further provides an electronic device, which may be the above-mentioned Figure 2 shown electronic device 200. The electronic device 200 includes a main board 210 and a voice processing device 100 fixed to the main board 210. The structure of the voice processing device 100 may be the structure of the voice processing device described in any of the above Figure 1 , Figure 7 and Figure 8 figures.

[0120] The embodiment of the present application further provides a computer-readable storage medium, in which computer program code is stored. When the processor 110 executes the computer program code, the processor 110 executes Figure 3 or Figure 4 the voice processing method shown.

[0121] The embodiment of the present application further provides a computer program product. When the processor 110 executes the computer program product, the processor 110 executes Figure 3 or Figure 4 the voice processing method shown.

[0122] The above detailed description of the voice method and the analysis of the beneficial effects can be correspondingly cited to the processor 110, the voice processing device 100, the electronic device 200, the computer-readable storage medium and the computer program product. The embodiments of the present application will not be repeated here.

[0123] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A voice processing method, characterized in that, Applied to a processor, the method includes: Obtain first voice data; Determine a feature sequence corresponding to the first voice data. When the feature sequence includes a preset feature and the preset feature is located at the end of the feature sequence, skip the execution of a first instruction corresponding to the first voice data.

2. The method according to claim 1, characterized in that, When the first voice data includes turning voice data corresponding to a turning word and the turning voice data is located at the end of the first voice data, the feature sequence includes a preset feature corresponding to the turning word, and the preset feature is located at the end of the feature sequence.

3. The method according to claim 1 or 2, characterized in that, After obtaining the first voice data, the method further includes: Obtain second voice data; When it is determined that the first voice data and the second voice data are complete voice data, execute a second instruction corresponding to the second voice data.

4. The method according to claim 3, characterized in that The execution of the second instruction corresponding to the second voice data includes: After obtaining the second voice data and when no voice data is obtained within a preset duration, execute the second instruction.

5. The method according to claim 3 or 4, characterized in that The execution of the second instruction corresponding to the second voice data includes: Obtain a first voiceprint feature corresponding to the voice data. When the similarity between the first voiceprint feature and a second voiceprint feature is greater than or equal to a preset value, execute the second instruction, where the second voiceprint feature is the voiceprint feature of a target user.

6. The method according to any one of claims 1-5, characterized in that, The determination of the feature sequence corresponding to the first voice data includes: Determine the feature sequence corresponding to the first voice data through Mel Frequency Cepstral Coefficients (MFCC).

7. A processor, characterized in that, The processor includes an interface circuit and a computing circuit that are coupled to each other; The interface circuit is configured to obtain first voice data; The computing circuit is configured to determine a feature sequence corresponding to the first voice data. When the feature sequence includes a preset feature and the preset feature is located at the end of the feature sequence, skip the execution of a first instruction corresponding to the first voice data.

8. The processor according to claim 7, wherein When the first voice data includes turning voice data corresponding to a turning word and the turning voice data is located at the end of the first voice data, the feature sequence includes a preset feature corresponding to the turning word, and the preset feature is located at the end of the feature sequence.

9. The processor according to claim 7 or 8, characterized in that, After the interface circuit obtains the first voice data; The interface circuit is further configured to obtain second voice data; The computing circuit is further configured to, when it is determined that the first voice data and the second voice data are complete voice data, execute a second instruction corresponding to the second voice data.

10. The processor according to claim 9, wherein: The computing circuit is specifically configured to, after obtaining the second voice data and when no voice data is obtained within a preset duration, execute the second instruction.

11. The processor according to claim 9 or 10, wherein: The computing circuit is specifically configured to obtain a first voiceprint feature corresponding to the voice data. When the similarity between the first voiceprint feature and a second voiceprint feature is greater than or equal to a preset value, execute the second instruction, where the second voiceprint feature is the voiceprint feature of a target user.

12. The processor according to any one of claims 7-11, wherein the computing circuit is specifically configured to determine the feature sequence corresponding to the first voice data through Mel Frequency Cepstral Coefficients (MFCC).

13. A voice processing device, characterized in that, The device includes a sound collector and a processor coupled to each other; the sound collector is configured to collect voice data, and the processor is the processor according to any one of claims 7-12.

14. The device according to claim 13, characterized in that, The device further includes a voice enhancement circuit coupled between the sound collector and the processor; the voice enhancement circuit is configured to perform voice enhancement processing on the voice data, and the voice enhancement processing includes at least one of acoustic echo cancellation (AEC), direction of arrival (DOA) source localization, blind source separation (BSS), dereverberation, or noise reduction.

15. The device according to claim 14, wherein the processor is further configured to perform noise reduction processing on the voice data after voice enhancement processing when the signal-to-noise ratio of the voice data after voice enhancement processing is less than a preset signal-to-noise ratio.

16. An electronic device, characterized in that, The electronic device includes a main board and a voice processing device fixed to the main board, and the voice processing device is the voice processing device according to any one of claims 13-15.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program runs on a processor, the processor is caused to execute the voice processing method according to any one of claims 1-6.

Images