Speech processing method and device

Abstract

According to the embodiment of the invention, a voice processing method and device, equipment and a medium are provided. The method comprises the steps that wake-up word voiceprint features of wake-up word voice segments are extracted, the wake-up word voice segments are matched with wake-up words, and the wake-up word voiceprint features indicate a target speaker; dividing the target voice into at least one voice segment, wherein the target voice is collected after the wake-up word voice segment; extracting at least one voiceprint feature of the at least one voice segment; based on the similarity between the at least one voiceprint feature and the wake-up word voiceprint feature, determining whether the at least one voice segment is from a target speaker; and determining instruction word recognition of the at least one voice segment based on a determination result of whether the at least one voice segment is from the target speaker. Therefore, noise interference or voice interference of others can be reduced in voice wake-up and voice control applications.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, computer-readable storage media, and computer program products for speech processing. Background Art

[0002] With the development of human-computer interaction technology, more and more voice control devices are appearing in people's lives, assisting in many aspects of life. For example, corresponding voice control devices can be deployed in various environments such as smart homes, smart travel (for example, smart cockpits), and smart factories. The intelligence of vehicle driving is also called a trend. Usually users will issue a wake-up word for the voice control device to wake up the voice control device. After the device is awakened, the corresponding operations can be completed according to the instructions issued by the user. Various sounds in the environment may affect the voice control of the device. Summary of the invention

[0003] In the first aspect of the present disclosure, a method for speech processing is provided. The method includes: extracting a wake-up word voiceprint feature of a wake-up word voice segment, the wake-up word voice segment matches the wake-up word, and the wake-up word voiceprint feature indicates a target speaker; dividing the target speech into at least one speech segment, and the target speech is collected after the wake-up word voice segment; extracting at least one voiceprint feature of at least one speech segment; based on the similarity between at least one voiceprint feature and the wake-up word voiceprint feature, respectively determining whether at least one speech segment is from the target speaker; and based on the determination result of whether at least one speech segment is from the target speaker, determining the instruction word recognition of at least one speech segment.

[0004] In the second aspect of the present disclosure, a device for speech processing is provided. The device includes: a wake-up word voiceprint extraction module, configured to extract the wake-up word voiceprint feature of the wake-up word voice segment, the wake-up word voice segment matches the wake-up word, and the wake-up word voiceprint feature indicates the target speaker; a voice segment division module, configured to divide the target speech into at least one voice segment, and the target speech is collected after the wake-up word voice segment; a voice segment voiceprint extraction module, configured to extract at least one voiceprint feature of at least one voice segment; a target speaker determination module, configured to determine whether at least one voice segment is from the target speaker based on the similarity between at least one voiceprint feature and the wake-up word voiceprint feature; and an instruction word recognition module, configured to determine the instruction word recognition of at least one voice segment based on the determination result of whether at least one voice segment is from the target speaker.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, which includes computer executable instructions, and when the computer executable instructions are executed by a processor, the method of the first aspect is implemented.

[0008] It should be understood that the contents described in the summary of the present invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 A schematic diagram showing an example architecture of a speech processing system according to some embodiments of the present disclosure;

[0012] Figure 3 A flowchart showing a process for speech processing according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A block diagram showing an apparatus for speech processing according to some embodiments of the present disclosure; and

[0014] Figure 5 A block diagram of a device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0016] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0017] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0018] The embodiments of the present disclosure may involve user data, data acquisition and / or use, etc. These aspects are subject to the corresponding laws, regulations and relevant provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user knows and confirms. Accordingly, when implementing each embodiment of the present disclosure, the type, scope of use, usage scenario, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with the relevant laws and regulations. The specific notification and / or authorization method can vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this respect.

[0019] The solutions described in this specification and in the examples, if they involve the processing of personal information, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0020] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. In this article, "model" may also be referred to as "machine learning model", "machine learning network" or "network", and these terms are used interchangeably in this article.

[0021] As briefly mentioned above, in the application of various voice control devices, various sounds in the environment may affect the voice control experience of the device. Take the smart cockpit as an example. In the smart cockpit, intelligent perception is an essential part of the smart cockpit and can also improve the user experience. For example, in a smart cockpit equipped with a voice wake-up and voice recognition system, the user can issue instructions to the car computer after voice wake-up, and the car computer can understand and execute the instructions. However, there will inevitably be various noise interferences in the in-car environment, which will lead to a decrease in the robustness of speech recognition for the target speaker who wakes up the cockpit.

[0022] Some smart cockpits have added the function of voice zone locking, the purpose of which is to only receive the voice in the voice collection area containing the wake-up word and suppress the voice in other voice collection areas, so as to achieve voice recognition for a specific voice collection area or a specific speaker. However, when other passengers, especially those in the back seat, speak across the voice collection area, voice zone locking will be very difficult. Moreover, voice zone locking itself has an error rate, which will result in low robustness of voice recognition of the target speaker, and the system may even recognize other people's voices as commands, which not only affects the experience but also threatens driving safety.

[0023] The embodiment of the present disclosure proposes a scheme for speech processing. According to various embodiments of the present disclosure, after detecting a wake-up word voice segment that matches the wake-up word, the wake-up word voiceprint feature of the wake-up word voice segment is extracted, and the wake-up word voiceprint feature indicates the target speaker of the wake-up device. The target voice collected after the wake-up word voice segment is divided into at least one voice segment, and at least one voiceprint feature of the at least one divided voice segment is extracted. Based on the similarity between at least one voiceprint feature and the wake-up word voiceprint feature, it is determined whether at least one voice segment comes from the target speaker. Based on the determination result of whether at least one voice segment comes from the target speaker, the instruction word recognition of at least one voice segment is determined. Thus, the voiceprint feature of the wake-up word can be used to identify the voice segment from the same speaker from the subsequent voice, and the voice segment from the same speaker can be recognized by instruction recognition, while the voice segment can be ignored. In this way, it can be confirmed that the recognized instruction word and the wake-up word are from the same speaker. In this way, noise interference or interference from other people's voices can be reduced in voice wake-up and voice control applications.

[0024] In the embodiments of the present disclosure, voiceprint feature information is introduced to improve the robustness of speech recognition of the target speaker when the device is voice-wake up in complex scenarios. The example embodiments of the present disclosure are described below in conjunction with the accompanying drawings. In some of the example embodiments below, in-vehicle smart devices, especially smart cockpits, are described as examples for the purpose of explanation only. However, it should be understood that the embodiments of the present disclosure can be applied to various types of voice control devices, including but not limited to smart home devices, smart industrial equipment, and the like.

[0025] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In the environment 100, the electronic device 110 may be configured to detect, extract, divide, recognize, and process speech.

[0026] In some embodiments, vehicle 130 may be any type of vehicle that can carry people and / or objects and move through a power system such as an engine, including but not limited to cars, trucks, buses, electric vehicles, RVs, etc. In some cases, vehicle 130 in environment 100 may be a comprehensive system that integrates functions such as environmental perception, planning and decision-making, and multi-level assisted driving. Such a vehicle is also called a smart car. Further, vehicle 130 may be an autonomous driving vehicle. In some embodiments, vehicle 130 may be equipped with an on-board device with voice control capability, such as a smart cockpit.

[0027] The environment 100 also includes one or more sound collection devices 120 (e.g., microphone arrays). The one or more sound collection devices 120 may be disposed in a vehicle 130 for collecting sounds. The collected sounds may include the voice of a target speaker 140 for voice control of the vehicle-mounted device, the voice of a passenger, the voice of a navigator, background music, vehicle noise, etc.

[0028] In some embodiments, the electronic device 110 may be a fixed device or a mobile device. In some embodiments, the electronic device 110 may be part of an intelligent control system or an intelligent cockpit system of the vehicle 130. In some embodiments, the electronic device 110 may be arranged in association with the vehicle 130, for example, may be placed in the vehicle 130. In some embodiments, the electronic device 110 may be arranged remotely relative to the vehicle 130.

[0029] The electronic device 110 may be any type of device or system with computing capabilities, including a terminal device or a server device. The terminal device may be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. The server device may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like.

[0030] It should be understood that the structure and functionality of environment 100 are described for exemplary purposes only and does not imply any limitation on the scope of the present disclosure.

[0031] Figure 2 A schematic diagram of an example architecture 200 of a speech processing system according to some embodiments of the present disclosure is shown. The example architecture 200 generally includes a speech wake-up model 230, a voiceprint extraction model 250, a voiceprint matching module 260, and an instruction word recognition module 270. The example architecture 200 may be partially or entirely implemented at the electronic device 110. For the purpose of explanation, the following reference Figure 1 An example architecture 200 is described.

[0032] The user voice 210 can be collected by one or more sound collection devices 120. In some embodiments, the example architecture 200 also includes an acoustic feature extraction model 220. The acoustic feature extraction model 220 is configured to receive the user voice 210 and convert it into an acoustic feature matrix. Acoustic features refer to physical quantities that identify the acoustic characteristics of speech, such as energy concentration areas, resonance peak frequencies, resonance peak strengths, etc. that represent timbre, as well as duration, fundamental frequency, voice power, etc. that represent the rhythmic characteristics of speech. For example, the acoustic feature extraction model 220 transforms the user voice 210 in a preset manner according to a preset feature type, and extracts the corresponding type of acoustic features from it. For another example, the acoustic feature extraction model 320 extracts acoustic features from speech based on a deep learning model. It should be understood that the present disclosure does not limit the implementation of the acoustic feature extraction model.

[0033] The voice wake-up model 230 is configured to receive an acoustic feature matrix and detect whether the text information corresponding to the acoustic feature matrix contains a wake-up word. A voice segment that matches the wake-up word is called a wake-up word voice segment. A wake-up word refers to a word used to wake up a voice control device. Such a wake-up word can be a preset specific word or a custom word. The voice wake-up model 230 can be a model based on pattern matching or a model based on a neural network. It should be understood that the present disclosure does not limit the implementation method of the voice wake-up model.

[0034] If the voice wake-up model detects a wake-up word from the user voice 210, the voice control device (e.g., an in-vehicle device) may be caused to enter a wake-up mode. In some embodiments, the wake-up mode of the voice control device may include a wake-up word continuous speaking (also called one-shot) mode, in which the collected user voice 210 may include a wake-up word and an instruction word. In some embodiments, the wake-up mode of the voice control device may include a normal wake-up mode, in which the collected user voice 210 contains a wake-up word. After the wake-up word is detected, a prompt voice, such as a voice segment of text-to-speech (TTS), may be played to prompt the target speaker 140 to continue to speak the instruction word. Therefore, the voice collected after the prompt voice is played may be determined as the target voice to be used for instruction word recognition. In some embodiments, it may be determined whether the duration of the voice (also called the first voice) collected together with the wake-up word voice segment exceeds a preset duration. If the duration of the first voice exceeds the preset duration, it is determined to enter the one-shot wake-up mode. At this time, the target voice to be used for instruction word recognition can be obtained by deleting the wake-up word voice segment from the first voice. If the duration of the first voice does not exceed the preset duration, it is determined to enter the normal wake-up mode. The speech collected after the prompt speech is played can be determined as the target speech to be used for instruction word recognition.

[0035] The voice wake-up model 230 can be woken up in real time and determine the wake-up word voice segment during the processing of the audio stream. Such a wake-up word voice segment can contain a complete wake-up word or only the first part of the wake-up word. For example, the complete wake-up word includes "Hello, XYZ". The user voice 210 includes "Hi, hello, XYZ, please open the window". When the voice wake-up model 230 detects "Hello, XY", it can determine that the target voice includes the wake-up word based on the matching probability, thereby immediately triggering the entry into the wake-up mode. In this way, a streaming voice wake-up effect can be achieved and the device wake-up efficiency can be improved.

[0036] In some embodiments, the sound collection environment where the voice control device is located can be divided into multiple voice collection areas (also called "sound zones"), and each voice collection area can be installed with one or more sound collection devices. The voice wake-up model 230 can determine the target voice collection area corresponding to the wake-up word voice segment from the voices collected in the multiple voice collection areas, and then obtain the target voice to be used for command word recognition from the sound collection device 120 in the voice collection area. Such a function is also called a sound zone locking function. For example, the in-car environment can be divided into four voice collection areas: the main driving area, the co-pilot area, the main driving rear area, and the co-pilot rear area. After the sound source of the user voice 210 is located by the closest sound collection device 120 in the car, the voice wake-up model 230 can lock its corresponding voice collection area and suppress the voice input of other voice collection areas, that is, the electronic device 110 only receives the voice input of the target voice collection area. On the one hand, this can reduce noise interference or intrusion of other passengers; on the other hand, this can provide accurate services for the target speaker 140. For example, the user voice 210 includes "Hi, hello, XYZ, please open the window." If the electronic device 110 determines that the user voice 210 comes from the co-pilot area, then when it is determined that the instruction needs to be executed, only the window of the co-pilot seat may be opened.

[0037] The voice wake-up model 230 inputs the wake-up word voice segment into the voiceprint extraction model 250. The voiceprint extraction model 250 is configured to extract the wake-up word voiceprint feature of the wake-up word voice segment. The wake-up word voiceprint feature contains a feature vector of the identity information of the target speaker 140. And, after determining to enter the wake-up mode, the target voice collected after the wake-up word voice segment is divided into at least one voice segment. Such voice segments are also input into the voiceprint extraction model 250. The voiceprint extraction model 250 is further configured to extract voiceprint features of these voice segments. The target voice collected after the wake-up word voice segment may include instructions issued by the target speaker who wakes up the device, and / or may include other human voices or noises in the environment. Therefore, after the target voice is divided into one or more voice segments, one or some of these voice segments may contain instruction words issued by the target speaker. Accordingly, the voiceprint features of these voice segments may also contain feature vectors of the identity information of the target speaker 140.

[0038] The granularity of the speech segments into which the target speech is divided depends on the duration of the target speech and the speech duration of the wake-up word. The smaller the granularity, the faster the response to the command word. In some embodiments, the voiceprint extraction module 250 can divide the target speech into at least one speech segment based on a preset window length and window shift by means of a sliding window. The preset window length is greater than or equal to the duration of the speech segment corresponding to the wake-up word. For example, the preset window length is 1-2 times the duration of the speech segment corresponding to the wake-up word. In this way, the effect of stream-like extraction of voiceprint features can be achieved. Additionally or alternatively, the speech segment of the last window length divided based on the target speech may not be calculated.

[0039] The target speech may contain non-speech segments. In order to improve the efficiency of speech processing, in some embodiments, the target speech is input into a voice activity detection (VAD) model 240. The voice activity detection model 240 is configured to perform endpoint detection on the target speech, identify and filter non-speech segments from the sound signal stream. As an example, the voice activity detection model 240 uses a trained neural network-based model to filter non-speech segments in the target speech so that the output speech contains valid speech parts. For example, for speech collected in a car, silent segments and vehicle noise segments can be filtered, thereby greatly reducing the length of the speech. The filtered valid speech output by the voice activity detection model 240 is further divided into at least one speech segment, which helps to save the computational workload of subsequent processing and improve the efficiency of voiceprint feature confirmation. One or more speech segments of the filtered target speech are provided to the voiceprint extraction model 250 for extracting corresponding voiceprint features.

[0040] The wake-up word voiceprint feature and the voiceprint features of the voice segments divided from the target speech are input to the voiceprint matching module 260. The voiceprint matching module 260 is configured to determine whether these voice segments are from the target speaker 140 based on the similarity between each voiceprint feature of each voice segment and the wake-up word voiceprint feature. For example, the target speech is divided into voice segment 1 and voice segment 2. If the similarity between the voiceprint feature of voice segment 1 and the wake-up word voiceprint feature exceeds the similarity threshold, it is determined that voice segment 1 comes from the target speaker 140. If the similarity between the voiceprint feature of voice segment 2 and the wake-up word voiceprint feature does not exceed the similarity threshold, it is determined that voice segment 2 does not come from the target speaker 140.

[0041] In some embodiments, the voiceprint features can be represented in the form of a multidimensional vector. The voiceprint matching module 260 can determine the cosine similarity between the voiceprint features of each voice segment and the wake-up word voiceprint features respectively as a measure of similarity. Cosine similarity, also known as cosine similarity, evaluates the similarity of two vectors by calculating the cosine value of the angle between them. If a voiceprint segment with a high similarity is found, it can be determined that the corresponding voice segment in the target voice belongs to the target speaker 140.

[0042] Further, the voiceprint matching module 260 provides the judgment result based on the similarity, that is, the judgment result of whether each voice segment in the target voice comes from the target speaker who said the wake-up word, to the instruction word recognition module 270. The instruction word recognition module 270 can directly receive the target voice from the voice wake-up model 230, or it can receive the voice segment divided after filtering the non-voice segment by the voice activity detection model 240. The instruction word recognition module 270 is configured to determine the instruction word recognition of these voice segments based on the determination result of whether one or more voice segments in the target voice come from the target speaker. If it is determined that a certain voice segment in these voice segments (voice segment 1 in the above example) comes from the target speaker 140 who issued the wake-up word, instruction word recognition is performed on the determined voice segment to identify the instruction word from the voice segment. Otherwise, if it is determined that a certain voice segment in these voice segments (voice segment 2 in the above example) does not come from the target speaker 140 who issued the wake-up word, instruction word recognition may not be performed on the voice segment. Thus, only the instruction word issued by the target speaker who wakes up the device can be recognized. In this way, the robustness of speech recognition of voice-awakening speakers in complex vehicle scenarios can be improved.

[0043] In some embodiments, the instruction word recognition module 270 may include a speech recognition (ASR) model 272 and a language model 274. The speech recognition model 272 is configured to perform speech recognition on the target speech to convert the speech into text. Further, the speech recognition model 272 may receive the determination result output by the voiceprint matching module 260 as to whether each speech segment in the target speech comes from the target speaker 140 to determine the timestamp of the speech segment belonging to the target speaker 140. The timestamp is aligned with the timestamp of the text converted thereto, and the speech recognition result, i.e., the text containing the instruction word of the target speaker 140, is output. The language model 274 is configured to receive such text and make a response after performing natural language understanding (NLP). For example, based on the instruction word recognition result of the speech segment belonging to the target speaker, the control operation of the voice control device can be determined. The corresponding control operation can be sent to the voice control device for execution.

[0044] Since the voiceprint extraction model 250 realizes stream-like voiceprint recognition by means of a sliding window, it can efficiently output the similarity calculation result after cooperating with the voiceprint matching module 260, thereby ensuring that the target speaker recognition result is output before the instruction word recognition module 270 outputs the instruction word recognition result. In this way, real-time performance can be improved.

[0045] In some embodiments, the speech recognition model 272 converts the received speech segments into text and sends it to the language model 274. The language model 274 receives the text and recognizes the instruction words contained therein. The language model 274 determines which of the recognized instruction words are from the target speaker 140 based on the similarity calculation results from the voiceprint matching module 260, and then executes the instruction words from the target speaker 140.

[0046] In some embodiments, if the speech recognition model 272 determines that a speech segment is from the target speaker 140, the text corresponding to the speech segment is sent to the language model 274. The language model 274 recognizes the instruction words from the text. If the speech recognition model 272 determines that a speech segment is not from the target speaker 140, the text corresponding to the speech segment is not sent to the language model 274.

[0047] The example architecture 200 is described above through multiple example embodiments. The voice wake-up model 230, the voice recognition model 272, and the voiceprint extraction model 250 can be trained in advance before application. In some embodiments, the pre-trained models of these models are fine-tuned using the instruction word dataset to obtain a better recognition rate.

[0048] In some embodiments, the voice wake-up model 230 , the voice activity detection model 240 , and the voice recognition model 272 are all streaming recognition models to ensure the real-time performance of voice processing.

[0049] In some embodiments, the wake-up word is a wake-up word for an in-vehicle device. The electronic device 110 may determine a control operation for the in-vehicle device based on the result of the command word recognition.

[0050] In summary, according to various embodiments of the present disclosure, by combining voice wake-up and voiceprint recognition, it is possible to determine whether the wake-up word voice and the command word voice are from the same speaker. This can reduce noise interference or interference from other people's voices, and complete robust and smooth control of the voice control device. In terms of specific implementation, a streaming voiceprint recognition method can be used to calculate the voiceprint similarity of the wake-up word voice segment with several voice segments. Voice segments that exceed the similarity threshold will be considered to be instructions from the target speaker, so that the voiceprint similarity results can be calculated before the command word recognition ends. Voice segments below the similarity threshold can be considered not to be from the target speaker and can be automatically ignored and discarded without the need for command word recognition.

[0051] Example Process

[0052] Figure 3 FIG. 3 is a flowchart of a process 300 for speech processing according to some embodiments of the present disclosure. The process 300 may be implemented at the electronic device 110. Figure 1 Process 300 is described.

[0053] In block 310, the electronic device 110 extracts a wake-up word voiceprint feature of the wake-up word speech segment. The wake-up word speech segment matches the wake-up word. The wake-up word voiceprint feature indicates a target speaker.

[0054] In block 320, the electronic device 110 divides the target voice into at least one voice segment. The target voice is collected after the wake-up word voice segment.

[0055] In block 330 , the electronic device 110 extracts at least one voiceprint feature of at least one speech segment.

[0056] In box 340, the electronic device 110 determines whether the at least one voice segment is from the target speaker based on the similarity between the at least one voiceprint feature and the wake-up word voiceprint feature.

[0057] In block 350 , the electronic device 110 determines instruction word recognition for the at least one speech segment based on the determination result of whether the at least one speech segment is from the target speaker.

[0058] In some embodiments, the electronic device 110 may divide the target speech into at least one speech segment based on a preset window length and a window shift. The preset window length is greater than or equal to the duration of the speech segment corresponding to the wake-up word.

[0059] In some embodiments, if the first similarity between the first voiceprint feature of at least one voiceprint feature and the wake-up word voiceprint feature exceeds a similarity threshold, the electronic device 110 determines that the first voice segment corresponding to the first voiceprint feature comes from the target speaker. If the second similarity between the second voiceprint feature of at least one voiceprint feature and the wake-up word voiceprint feature does not exceed the similarity threshold, the electronic device 110 determines that the second voice segment corresponding to the second voiceprint feature does not come from the target speaker.

[0060] In some embodiments, if the electronic device 110 determines that the first voice segment in at least one voice segment is from the target speaker, the electronic device 110 recognizes the instruction word from the first voice segment; and if it determines that the second voice segment in at least one voice segment is not from the target speaker, the electronic device 110 abandons the recognition of the instruction word for the second voice segment.

[0061] In some embodiments, if the duration of the first voice collected together with the wake-up word voice segment exceeds the preset duration, the electronic device 110 obtains the target voice by deleting the wake-up word voice segment from the first voice. If the duration of the first voice does not exceed the preset duration, the electronic device 110 determines the voice collected after the prompt voice is played as the target voice.

[0062] In some embodiments, the electronic device 110 may determine a target voice collection area corresponding to the wake-up word voice segment from multiple voice collection areas; and obtain the target voice from a sound collection device in the voice collection area.

[0063] In some embodiments, the electronic device 110 may filter non-voice segments from the target voice by performing voice activity detection on the target voice to obtain a filtered target voice; and divide the filtered target voice into at least one voice segment.

[0064] In some embodiments, the wake-up word is a wake-up word for an in-vehicle device, and the electronic device 110 can determine a control operation for the in-vehicle device based on the result of the command word recognition.

[0065] Example devices and equipment

[0066] Figure 4 1 shows a schematic structural block diagram of an apparatus 400 for autonomous driving according to certain embodiments of the present disclosure. The apparatus 400 may be implemented as or included in the electronic device 110. Each module / component in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0067] As shown in the figure, the device 400 includes a wake-up word voiceprint extraction module 410, which is configured to extract the wake-up word voiceprint feature of the wake-up word voice segment, the wake-up word voice segment matches the wake-up word, and the wake-up word voiceprint feature indicates the target speaker. The device 400 also includes a voice segment division module 420, which is configured to divide the target speech into at least one voice segment, and the target speech is collected after the wake-up word voice segment. The device 400 also includes a voice segment voiceprint extraction module 430, which is configured to extract at least one voiceprint feature of at least one voice segment. The device 400 also includes a speaker determination module 440, which is configured to determine whether at least one voice segment is from the target speaker based on the similarity between at least one voiceprint feature and the wake-up word voiceprint feature. The device 400 also includes an instruction word recognition module 450, which is configured to determine the instruction word recognition of at least one voice segment based on the determination result of whether at least one voice segment is from the target speaker.

[0068] In some embodiments, the speech segment division module 420 is configured to divide the target speech into at least one speech segment based on a preset window length and a window shift, and the preset window length is greater than or equal to the duration of the speech segment corresponding to the wake-up word.

[0069] In some embodiments, the speaker determination module 440 is configured to: determine that a first speech segment corresponding to the first voiceprint feature comes from the target speaker if a first similarity between a first voiceprint feature among at least one voiceprint feature and the wake-up word voiceprint feature exceeds a similarity threshold; and determine that a second speech segment corresponding to the second voiceprint feature does not come from the target speaker if a second similarity between a second voiceprint feature among at least one voiceprint feature and the wake-up word voiceprint feature does not exceed a similarity threshold.

[0070] In some embodiments, the instruction word recognition module 450 is configured to: if it is determined that a first speech segment in at least one speech segment is from a target speaker, recognize instruction words from the first speech segment; and if it is determined that a second speech segment in at least one speech segment is not from the target speaker, abandon instruction word recognition for the second speech segment.

[0071] In some embodiments, the device 400 also includes a deletion module, which is configured to: if the duration of the first voice collected together with the wake-up word voice segment exceeds a preset duration, obtain the target voice by deleting the wake-up word voice segment from the first voice; and a target voice determination module, which is configured to determine the voice collected after the prompt voice is played as the target voice if the duration of the first voice does not exceed a preset duration.

[0072] In some embodiments, the device 400 also includes a target voice collection area determination module, which is configured to determine a target voice collection area corresponding to a wake-up word voice segment from multiple voice collection areas; and a target voice acquisition module, which is configured to obtain a target voice from a sound collection device in the voice collection area.

[0073] In some embodiments, the speech segment division module 420 is configured to filter non-speech segments from the target speech by performing speech activity detection on the target speech to obtain a filtered target speech; and divide the filtered target speech into at least one speech segment.

[0074] In some embodiments, the wake-up word is a wake-up word for an in-vehicle device, and the apparatus 400 further includes a control operation determination module configured to determine a control operation for the in-vehicle device based on a result of the instruction word recognition.

[0075] Figure 5 A block diagram of an electronic device 500 is shown in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 5 The electronic device 500 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 5 The electronic device 500 shown can be used to implement Figure 1 The electronic device 110. The electronic device 500 may include or be implemented as Figure 4 Device 400.

[0076] like Figure 5 As shown, the electronic device 500 is in the form of a general electronic device. The components of the electronic device 500 may include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 520. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0077] The electronic device 500 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the electronic device 500.

[0078] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 5 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 520 may include a computer program product 525 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0079] The communication unit 540 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0080] The input device 550 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 560 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 may also communicate with one or more external devices (not shown) through the communication unit 540 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 500, or communicate with any device that allows the electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0081] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0082] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0083] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0084] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0085] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0086] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for speech processing, comprising: Extracting a wake-up word voiceprint feature of a wake-up word speech segment, wherein the wake-up word speech segment matches the wake-up word, and the wake-up word voiceprint feature indicates a target speaker; Dividing the target speech into at least one speech segment, wherein the target speech is collected after the wake-up word speech segment; Extracting at least one voiceprint feature of the at least one speech segment; Based on the similarity between the at least one voiceprint feature and the wake-up word voiceprint feature, determining whether the at least one voice segment comes from the target speaker; as well as Based on the determination result of whether the at least one speech segment comes from the target speaker, instruction word recognition for the at least one speech segment is determined.

2. The method according to claim 1, wherein dividing the target speech into the at least one speech segment comprises: Dividing the target speech into the at least one speech segment based on a preset window length and a window shift, The preset window length is greater than or equal to the duration of the voice segment corresponding to the wake-up word.

3. The method of claim 1 , wherein determining whether the at least one speech segment is from the target speaker comprises: If a first similarity between a first voiceprint feature of the at least one voiceprint feature and the wake-up word voiceprint feature exceeds a similarity threshold, determining that a first speech segment corresponding to the first voiceprint feature comes from the target speaker; and If the second similarity between a second voiceprint feature of the at least one voiceprint feature and the wake-up word voiceprint feature does not exceed the similarity threshold, it is determined that the second speech segment corresponding to the second voiceprint feature is not from the target speaker.

4. The method of claim 1, wherein determining instruction word recognition for the at least one speech segment comprises: If it is determined that a first speech segment of the at least one speech segment is from the target speaker, identifying an instruction word from the first speech segment; as well as If it is determined that a second speech segment in the at least one speech segment does not come from the target speaker, instruction word recognition for the second speech segment is abandoned.

5. The method according to claim 1, further comprising: If the duration of the first voice collected together with the wake-up word voice segment exceeds a preset duration, obtaining the target voice by deleting the wake-up word voice segment from the first voice; as well as If the duration of the first voice does not exceed the preset duration, the voice collected after the prompt voice is played is determined as the target voice.

6. The method according to claim 1, further comprising: Determine a target voice collection area corresponding to the wake-up word voice segment from multiple voice collection areas; as well as The target voice is obtained from a sound collecting device in the voice collecting area.

7. The method according to claim 1, wherein dividing the target speech into the at least one speech segment comprises: filtering non-speech segments from the target speech by performing voice activity detection on the target speech to obtain a filtered target speech; as well as The filtered target speech is divided into the at least one speech segment.

8. The method according to claim 1, wherein the wake-up word is a wake-up word of an in-vehicle device, and the method further comprises: Based on the result of the instruction word recognition, a control operation on the in-vehicle device is determined.

9. A device for speech processing, comprising: a wake-up word voiceprint extraction module, configured to extract a wake-up word voiceprint feature of a wake-up word speech segment, the wake-up word speech segment matches the wake-up word, and the wake-up word voiceprint feature indicates a target speaker; A voice segment division module, configured to divide the target voice into at least one voice segment, wherein the target voice is collected after the wake-up word voice segment; A speech segment voiceprint extraction module, configured to extract at least one voiceprint feature of the at least one speech segment; a speaker determination module, configured to determine whether the at least one voice segment comes from the target speaker based on the similarity between the at least one voiceprint feature and the wake-up word voiceprint feature; as well as The instruction word recognition module is configured to determine instruction word recognition for the at least one speech segment based on a determination result of whether the at least one speech segment comes from the target speaker.

10. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 8 when executed by the at least one processing unit.

11. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 8.

12. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 8.

Images