Audio transmission method, control device and terminal device

Abstract

This application provides an audio transmission method, control device, and terminal device in several embodiments. The method receives a voice data stream input by a user. Responding to the voice data stream, which includes multiple voice data blocks, the method identifies feature words within the voice data blocks. These feature words include a first feature word and a second feature word. The input time of the voice corresponding to the first feature word precedes the input time of the voice corresponding to the second feature word. If the first feature word is identified in the voice data block, a recognition success flag is set for the voice data stream, and the voice data stream with the recognition success flag is sent to the terminal device, enabling the terminal device to identify the second feature word within the voice data stream. This application achieves rapid response in both the control device and the terminal device, shortening wake-up time and improving wake-up performance. Therefore, this application not only efficiently and quickly completes audio transmission and secondary verification by the terminal device but also enhances the user experience.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to an audio transmission method, control device, and terminal device. Background Technology

[0002] Terminal devices refer to electronic devices capable of outputting specific display images, such as smart TVs, mobile terminals, smart advertising screens, and projectors. With the rapid development of terminal devices, their functions will become increasingly rich and their performance increasingly powerful, enabling two-way human-computer interaction and integrating multiple functions such as audio-visual, entertainment, and data processing to meet diverse and personalized user needs.

[0003] With the development of terminal devices, people's requirements for them are also increasing. Manual operation can no longer meet the need for convenience and speed. More and more terminal devices are controlled by voice. However, voice control is always in a waiting state, resulting in high power consumption of terminal devices. This highlights the importance of wake word recognition. Wake word recognition is a branch of speech recognition tasks. It requires detecting a finite number of predefined activation words or keywords from a speech stream, without recognizing all the speech. Wake words are pre-set, and generally consist of four characters. The more syllables covered and the greater the syllable differences, the better the wake-up performance and the lower the false wake-up performance.

[0004] However, in existing wake-word-based speech recognition transmission methods, the target speaker needs to utter the complete wake-word before the fully saved audio is transmitted. This results in a lengthy wake-up process, hindering efficient and rapid speech transmission and leading to a poor user experience. Summary of the Invention

[0005] This invention provides an audio transmission method, a terminal device, and the terminal device itself. It addresses the problem that the entire wake-up process is too long when a user initiates a wake-up command, making it impossible to efficiently and quickly complete voice transmission.

[0006] In a first aspect, some embodiments of this application provide a control device, the control device comprising:

[0007] The communication module is configured to communicate with the terminal device.

[0008] The control module is configured as follows:

[0009] Receives voice data streams input by the user;

[0010] In response to the voice data stream, the voice data stream includes a plurality of voice data blocks;

[0011] Feature words are identified in the speech data block, and the feature words include at least one keyword in the voice interaction wake-up word; the feature words include a first feature word and a second feature word; the input time of the speech corresponding to the first feature word is earlier than the input time of the speech corresponding to the second feature word;

[0012] If the first feature word is identified in the speech data block, a recognition success flag is set for the speech data stream, and the recognition success flag includes the first feature word;

[0013] The voice data stream carrying the recognition success flag is sent to the terminal device so that the terminal device can recognize the second feature word in the voice data stream.

[0014] Secondly, some embodiments of this application provide a terminal device, the terminal device comprising:

[0015] The communicator is configured to communicate with the control equipment.

[0016] The controller is configured as follows:

[0017] Acquire a voice data stream, the voice data stream including multiple voice data blocks; the voice data stream is streaming media data sent by the terminal device when it recognizes a first feature word in the voice data block, the voice data stream has a recognition success flag, and the recognition success flag includes the first feature word;

[0018] A second feature word is identified in the speech data stream, wherein the input time of the speech corresponding to the first feature word is earlier than the input time of the speech corresponding to the second feature word.

[0019] Thirdly, some embodiments of this application provide an audio transmission method, the method comprising:

[0020] The control device receives the user's voice data stream;

[0021] In response to the voice data stream, which includes multiple data blocks, and identifying feature words in the voice data blocks, the feature words including at least one keyword in a voice interaction wake-up word; the feature words including a first feature word and a second feature word; the input time of the voice corresponding to the first feature word is earlier than the input time of the voice corresponding to the second feature word;

[0022] If the first feature word is identified in the speech data block, a recognition success flag is set for the speech data stream, and the recognition success flag includes the first feature word;

[0023] The voice data stream carrying the recognition success flag is sent to the terminal device so that the terminal device can recognize the second feature word in the voice data stream.

[0024] As can be seen from the above technical solutions, some embodiments of this application provide an audio transmission method, a control device, and a terminal device. By receiving a user-input voice data stream, which includes multiple voice data blocks, feature words are identified within the voice data blocks. These feature words include a first feature word and a second feature word. The input time of the voice corresponding to the first feature word precedes the input time of the voice corresponding to the second feature word. If the first feature word is identified in the voice data block, a recognition success flag is set for the voice data stream, and the voice data stream with the recognition success flag is sent to the terminal device, enabling the terminal device to identify the second feature word within the voice data stream. This application achieves rapid response from the terminal device, shortening wake-up time and improving wake-up performance. This application not only efficiently and quickly completes audio transmission and secondary verification of the terminal device but also enhances the user experience. Attached Figure Description

[0025] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic diagram of the system architecture of an audio transmission method, a control device 100, and a terminal device 200 according to some embodiments is shown.

[0027] Figure 2 A hardware configuration block diagram of a control device 100 according to some embodiments is shown;

[0028] Figure 3 A hardware configuration block diagram of a terminal device 200 according to some embodiments is shown;

[0029] Figure 4 A software configuration diagram of a terminal device 200 according to some embodiments is shown;

[0030] Figure 5 A schematic diagram of a voice interaction network architecture according to some embodiments is shown;

[0031] Figure 6 This diagram illustrates the state of a terminal device 200 after its user interface is woken up, according to some embodiments.

[0032] Figure 7 A schematic diagram of a wake-up process for a control device 100 and a terminal device 200 according to some embodiments is shown;

[0033] Figure 8 A schematic diagram of a wake-up process for a control device 100 and a terminal device 200 according to some embodiments is shown;

[0034] Figure 9 A schematic diagram of the wake-up process of another control device 100 and terminal device 200 according to some embodiments is shown;

[0035] Figure 10 This diagram illustrates the training and learning process of the feature word recognition model according to some embodiments;

[0036] Figure 11 A schematic diagram is shown illustrating the calculation of wake-up time using an existing wake-up process according to some embodiments;

[0037] Figure 12 A schematic diagram is shown illustrating the calculation of wake-up time using the wake-up process provided in some embodiments of this application. Detailed Implementation

[0038] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0039] Based on the exemplary embodiments described in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the appended claims. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can constitute a complete implementation on its own. It should be noted that the brief descriptions of terminology in this application are merely for the convenience of understanding the embodiments described below, and are not intended to limit the implementation of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0040] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0041] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0042] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0043] Figure 1 An exemplary system architecture is shown that can be applied to the audio transmission method, control device, and terminal device of this application. For example... Figure 1 As shown, 10 represents a server, 100 represents a control device, and 200 represents a terminal device, exemplarily including (smart TV 200a, mobile device 200b, smart speaker 200c). The control device 100 can be a remote control, and communication between the remote control and the terminal device 200 includes infrared protocol communication, Bluetooth protocol communication, wireless or other wired methods to control the terminal device 200. Users can control the terminal device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0044] In this application, server 10 and terminal device 200 communicate data through various communication methods. Terminal device 200 can be connected via local area network (LAN), wireless local area network (WLAN), and other networks. Server 10 can provide various content and interactive features to terminal device 200. For example, terminal device 200 and server 10 can send and receive information, and receive software updates.

[0045] Server 10 can be a server that provides various services, such as a backend server that supports audio data collected by terminal device 200. The backend server can analyze and process the received audio and other data, and feed back the processing results (such as endpoint information) to terminal device 200. Server 10 can be a server cluster or multiple server clusters, and can include one or more types of servers.

[0046] The terminal device 200 can be either hardware or software. When the terminal device 200 is hardware, it can be various electronic devices with sound acquisition capabilities, including but not limited to smart speakers, smartphones, televisions, tablets, e-book readers, smartwatches, media players, computers, AI devices, robots, smart vehicles, etc. When the terminal device 200 is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software programs or software modules (e.g., used to provide sound acquisition services) or as a single software program or software module. No specific limitations are imposed here.

[0047] It should be noted that the audio transmission method provided in this application embodiment can be executed by server 10, by terminal device 200, or by both server 10 and terminal device 200. This application does not limit this.

[0048] Figure 2 An exemplary configuration block diagram of a control device 100 according to an exemplary embodiment is shown. Figure 2 As shown, the control device 100 includes a control module 110, a communication module 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input operation commands and convert them into commands that the terminal device 200 can recognize and respond to, acting as an intermediary between the user and the terminal device 200. The communication module 130 is used for external communication and includes at least one of a WIFI chip, a Bluetooth module, NFC, or a replaceable module. The user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, buttons, or a replaceable module.

[0049] Figure 3 A hardware configuration block diagram of a terminal device 200 according to an exemplary embodiment is shown. For example... Figure 3 The terminal device 200 shown includes at least one of a communicator 220, a detector 230, an external device interface 240, a controller 250, an audio output interface 260, a memory, a power supply, and a user interface 270. The controller 250 includes a central processing unit, an audio processor, RAM, ROM, and a first to an nth interface for input / output.

[0050] The communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator 220 may include at least one of the following: a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The terminal device 200 can establish the transmission and reception of control signals and data signals through the communicator 220 and the server 10.

[0051] The user interface can be used to receive external control signals.

[0052] Detector 230 is used to collect signals from the external environment or to interact with the external environment. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0053] The sound acquisition device can be a microphone, also known as a "microphone" or "voice transducer," which can be used to receive the user's voice and convert the sound signal into an electrical signal. The terminal device 200 can be equipped with at least one microphone. In some embodiments, the terminal device 200 can be equipped with two microphones, which, in addition to acquiring sound signals, can also perform noise reduction. In other embodiments, the terminal device 200 can also be equipped with three, four, or more microphones, enabling sound signal acquisition, noise reduction, sound source identification, and directional recording functions, etc.

[0054] Furthermore, the microphone can be built into the terminal device 200, or it can be connected to the terminal device 200 via wired or wireless means. Of course, this embodiment does not limit the location of the microphone on the terminal device 200. Alternatively, the terminal device 200 may not include a microphone, meaning the microphone is not located within the terminal device 200. The terminal device 200 can connect an external microphone (also called a microphone) via an interface (such as USB interface 130). This external microphone can be fixed to the terminal device 200 using an external fastener (such as a camera bracket with a clip).

[0055] The controller 250 controls the operation of the terminal device 200 and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the terminal device 200.

[0056] For example, the controller includes at least one of a central processing unit (CPU), an audio processor, RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0057] In some examples, the operating system of terminal device 200 is Android, such as... Figure 4 As shown, the smart TV 200-1 can be logically divided into an application layer (referred to as the "application layer") 21, a kernel layer 22, and a hardware layer 23.

[0058] Among them, such as Figure 4 As shown, the hardware layer may include Figure 2 The controller 250, communicator 220, detector 230, etc., are shown. Application layer 21 includes one or more applications. These applications can be system applications or third-party applications. For example, application layer 21 includes a voice recognition application, which can provide a voice interaction interface and services to enable the connection between the smart TV 200-1 and the server 10.

[0059] The kernel layer 22 serves as a software middleware between the hardware layer and the application layer 21, and is used to manage and control hardware and software resources.

[0060] In some examples, kernel layer 22 includes a detector driver that sends voice data collected by detector 230 to a speech recognition application. For instance, when the speech recognition application in terminal device 200 starts and a communication connection is established between terminal device 200 and server 10, the detector driver sends user-input voice data collected by detector 230 to the speech recognition application. The speech recognition application then sends query information containing this voice data to intent recognition module 202 in the server. Intent recognition module 202 inputs the voice data sent by terminal device 200 into the intent recognition model.

[0061] To clearly illustrate the embodiments of this application, the following description is provided in conjunction with... Figure 5 This application describes a speech recognition network architecture provided in its embodiments.

[0062] See Figure 5 , Figure 5 This is a schematic diagram of a voice interaction network architecture provided in an embodiment of this application. Figure 5 In this system, the terminal device receives input information and outputs the processing results of that information. The speech recognition module deploys a speech recognition service to convert audio into text; the semantic understanding module deploys a semantic understanding service to perform semantic parsing on the text; the business management module deploys a business instruction management service to provide business instructions; the language generation module deploys a natural language generation (NLG) service to convert instructions instructing the terminal device to execute into text; and the speech synthesis module deploys a text-to-speech (TTS) service to process the text corresponding to the instructions and send it to the speaker for playback. In one embodiment, Figure 5 The architecture shown can contain multiple entity service devices with different business services deployed, or one or more entity service devices can combine one or more functional services.

[0063] In some embodiments, the following describes the basis Figure 5 The process of processing information from input terminal devices in the architecture shown is described with an example, taking a query statement input by voice from input terminal device 200 as an example:

[0064] [Speech Recognition]

[0065] After receiving a query statement input via voice, the terminal device 200 can perform noise reduction processing and feature extraction on the audio of the query statement. The noise reduction processing may include steps such as removing echoes and environmental noise.

[0066] [Semantic understanding]

[0067] Using acoustic and language models, natural language understanding is performed on the identified candidate text and associated contextual information. The text is parsed into structured, machine-readable information, including business domain, intent, slots, and other semantic information. An executable intent confidence score is obtained, and the semantic understanding module selects one or more candidate executable intents based on the determined intent confidence score.

[0068] [Business Management]

[0069] Based on the semantic parsing results of the query statement text, the semantic understanding module sends query instructions to the corresponding business management module to obtain the query results provided by the business service, as well as the actions required to "complete" the user's final request, and feeds back the device execution instructions corresponding to the query results.

[0070] It should be noted that, Figure 5 The architecture shown is merely an example and is not intended to limit the scope of protection of this application. Other architectures can also be used to achieve similar functions in the embodiments of this application. For example, all or part of the above process can be completed by a smart terminal, which will not be elaborated here.

[0071] The above embodiments describe the hardware / software architecture of the terminal device 200, the control device 100, and the functional implementation. With the continuous development of speech recognition technology and smart homes, the requirements for user experience in human-computer interaction are becoming increasingly demanding. Voice wake-up is a branch of speech recognition tasks, requiring the detection of a limited number of predefined activation words or keywords from a stream of speech, without needing to recognize all the speech. This type of technology can be applied to various fields, such as mobile phones, smart speakers, robots, smart homes, in-vehicle devices, and wearable terminal devices. In some scenarios, if a user needs to control the terminal device 200 via voice to perform related operations, the user needs to say a complete wake-up word, such as "Friend, come out," "Hello, Tiantian," or "Hello, classmate," etc., and then confirm the wake-up word "Friend, come out" before the voice interaction function on the terminal device 200 can be activated. During the entire wake-up process, the terminal device 200 needs to recognize the complete wake-up word, such as "Friend, come out," to activate the voice wake-up function. Therefore, the entire wake-up process is relatively long, which is detrimental to the user experience.

[0072] To improve the wake-up time efficiency of the terminal device 200 and quickly complete the audio transmission between the control device 100 and the terminal device 200, this application embodiment provides an audio transmission between the control device 100 and the terminal device 200 to improve wake-up performance and shorten wake-up time. In some embodiments, the control device 100 can receive the user's input voice wake-up word under any circumstances. For example, when the user says a wake-up word, such as "friends come out," "friends come out" is the voice wake-up word, or it can be "hi TV," "hi TV" is the voice wake-up word. The voice wake-up word can be a pre-set wake-up word of the terminal device 200 or a non-pre-set wake-up word. The user can set it themselves, such as "I love my home," "puppy TV," or "hey friends," etc. The user can also arbitrarily modify the wake-up word and the number of words in the wake-up word during use. If the number of bytes set in the wake-up word is too large, the wake-up recognition time will be too long. Therefore, most Chinese wake-up words use a combination of four characters, because the more syllables covered and the greater the syllable differences, the better the wake-up and false wake-up performance.

[0073] In some embodiments, the control device 100 acquires a voice data stream emitted by a user. This voice data stream is obtained by acquiring an audio signal, which is generated through streaming microphone recording. For example, when a user speaks, their voice is picked up by the microphone on the control device 100 in streaming mode, thus generating an audio signal. The final wavelength of the audio signal depends on the duration of the user's speech. For instance, if a user says "Friend, come out," and the speech lasts for one second, the length of the audio signal is the wavelength generated in that one second. The audio signal is then segmented at preset time intervals. These intervals can be set manually, but are typically 20ms intervals. Multiple voice data blocks are generated after segmentation. For example, segmenting a one-second audio signal at 20ms intervals yields 50 voice data blocks. These 50 voice data blocks are combined in the order of their segmentation to form the voice data stream. In other words, the voice data stream comprises multiple voice data blocks.

[0074] In some embodiments, feature words are identified in the voice data block. These feature words include at least one keyword from the voice interaction wake-up word. For example, if the voice interaction wake-up word is "Friend, come out," and the user's voice to the control device 100 is "Hello, friend," the feature word "friend" can be identified from the four characters "Hello, friend," and the control device 100 begins transmitting the voice data stream to the terminal device 200. This illustrates that in this embodiment, at least one keyword from the wake-up word needs to be identified in the feature words for voice data stream transmission between the control device 100 and the terminal device 200 to proceed.

[0075] In some embodiments, if a feature word is identified in the voice data block, the voice data stream is sent to the terminal device 200. For example, when a user says, "Friends, come out! How's the weather today?", the feature word "friends, come out" needs to be fully identified before the voice data stream containing "friends, come out" can be sent to the terminal device 200 for storage. The terminal device 200 then checks the stored voice data stream to determine if it contains the wake word "friends, come out," i.e., performs a secondary verification. If the verification is successful and the wake word "friends, come out" is indeed present, the terminal device 200 responds to the voice interaction wake-up command in the voice data stream. At this time, such as Figure 6 The diagram shown illustrates the state of a terminal device 200 after its user interface is woken up, according to an embodiment of this application. The terminal device 200 will provide feedback, including: animated displays, and a notification sound such as "Hi, how can I help you?". Figure 7 The diagram shows the existing wake-up process. This wake-up process relies on the user saying the wake-up word "like a friend coming out" before the voice data stream is saved and transmitted. The time it takes for the user to say the wake-up word, the time for wake-up word recognition, the time for transmitting the voice data stream, the time for the terminal device 200 to perform secondary verification, and the time for the terminal device 200 to finally pop up the wake-up interface all add up. It can be seen that the entire wake-up process takes a long time, the wake-up response time is also very slow, and the user experience is poor.

[0076] In some embodiments, feature words are identified in speech data blocks. Since there are multiple speech data blocks in the speech data stream, feature words can also be identified sequentially in multiple speech data blocks according to time order. If a feature word is identified in any speech data block, the control device 100 sends the speech data block with the identified feature word, as well as the remaining speech data blocks after the speech data block with the identified feature word, to the terminal device 200. For example, if there are five speech data blocks in the speech data stream, and no feature word is identified in the first speech data block during the speech data block identification process, the control device 100 does not send the speech data block to the terminal device 200, but identifies the subsequent speech data blocks sequentially according to time order. If a feature word is identified in the second speech data block, the control device 100 immediately sends the second speech data block and the remaining three speech data blocks to the terminal device 200.

[0077] In some embodiments, there is another scenario where no feature words are identified in any of the five voice data blocks mentioned above. In this case, the control device 100 does not send a voice data stream to the terminal device 200. This saves the terminal device 200 a significant amount of unnecessary wake-up recognition time, enabling it to complete the wake-up voice interaction function more quickly.

[0078] In some embodiments, the feature words include a first feature word and a second feature word. For example, if "friends come out" is a feature word, then the first feature word is "friends" and the second feature word is "come out". Furthermore, the input time of the speech corresponding to the first feature word precedes the input time of the speech corresponding to the second feature word. When the speech data stream is transmitted to the terminal device 200, the speech data stream corresponding to the first feature word is input to the terminal device 200 earlier than the speech data stream corresponding to the second feature word. For example, the terminal device 200 receives the speech data stream corresponding to the first feature word "friends" first, and then receives the speech data stream corresponding to the second feature word "come out".

[0079] like Figure 8 The diagram illustrates the wake-up process of a control device 100 and a terminal device 200 according to an embodiment of this application. By recognizing a first feature word in a voice data block, if the first feature word "friend" is recognized in the voice data block, the control device 100 sets a recognition success flag for the voice data stream. That is, the recognition success flag includes the first feature word. For example, when the word "friend" is recognized, the control device 100 immediately sends the voice data stream of "friend" to the terminal device 200, without waiting for the user to say the complete wake-up phrase "friend, come out." At this time, the control device 100 continues to recognize a second feature word in the voice data block. If the second feature word "come out" is recognized, the control device 100 also sends the voice data stream of "come out" to the terminal device 200. The terminal device 200 performs a secondary verification on the received first and second feature words to determine if it is the wake-up phrase "friend, come out." If it is "friend, come out," the terminal device 200 can then activate the voice interaction function.

[0080] In some embodiments, since the first feature word "friend" already has an identification success flag, the terminal device 200 does not need to perform a secondary verification to determine whether it is the first feature word "friend". The terminal device 200 only needs to perform a wake-up recognition on the second feature word "come out" in the voice data block. If the recognized second feature word is not "come out", the terminal device 200 will not give any response, and the user interface will not show any response, indicating that the voice interaction function has not been successfully awakened. Of course, the terminal device 200 can also perform a secondary verification on "friend come out" to ensure the accuracy of the wake-up recognition. In some embodiments of the present application, during the entire process of awakening the voice interaction function, it is possible to perform wake-up recognition while transmitting the voice data stream, so as to save time and greatly improve the user experience.

[0081] In some embodiments, such as Figure 9 shown, in order to further shorten the wake-up response time between the control device 100 and the terminal device 200 and further improve the wake-up efficiency. The control device 100 can also split the wake-up word into multiple keywords. For example, if the wake-up word is "hello friend", it can be split into "you", "good", "friend", and "companion", these four keywords. The control device 100 recognizes the first keyword in the wake-up word in the voice data block. If it is the keyword "you", the control device 100 transmits the voice data stream to the terminal device 200 wirelessly. The terminal device 200 then performs wake-up recognition on the remaining keywords "good", "friend", and "companion" in the voice data stream transmitted by the control device 100. If these three keywords are "good", "friend", and "companion", the terminal device 200 wakes up the voice interaction function. If the recognized keyword is not one of the three keywords "good", "friend", and "companion", then the terminal device 200 will not wake up the voice interaction function.

[0082] In some embodiments, such as Figure 10 shown, the feature word recognition model provided by the embodiments of the present application recognizes feature words in the voice data block by calling the feature word recognition model. Among them, the feature word recognition model is a neural network model obtained through training and learning based on a large number of pre-collected voice sample data. When training the voice sample data, it can be trained and learned in various forms. For example, a training mode with two feature words as a group can be selected for voice sample training, or one feature word can be used as a group, and the remaining feature words can be used as a group for voice sample training and learning.

[0083] For example, Figure 10The underlying data is the audio sample data of "Friend, come out". Neural network learning means inputting the audio sample data spoken by the user into the neural network for learning in groups of two feature words. After neural network learning, "Friend, come out" is classified and output as "Friend" and "come out". Or it can also be training and learning with the first character of the feature words as one group and the remaining characters as another group. For example, the underlying data is still the audio sample data of "Friend, come out". Similarly, the audio sample data spoken by the user is input into the neural network with the first character as one group and the remaining three characters as another group for learning. After neural network learning, "Friend, come out" is classified and output as "Peng" and "You come out". Through the setting inside the feature word recognition model, when performing feature word recognition, the corresponding feature word recognition model can be selected. For example, when recognizing the feature word "Friend", it belongs to the form of two feature words as one group, so the feature word recognition model that conducts speech training and completes learning with two feature words as one group can be correspondingly selected. Thus, more accurate wake-up recognition can be achieved, which can not only reduce the false wake-up rate but also improve the time efficiency of wake-up recognition.

[0084] In some embodiments, since the speech data block is generated from the audio signal, it is necessary to convert the speech data block into text information, that is, convert the audio signal into text. For example, the character "Peng" corresponds to an audio signal, and the corresponding audio signal also corresponds to a speech data block. Since the speech data block is generated from the audio signal and cannot be directly input into the feature word recognition model, it is necessary to convert the speech data block into text form. The converted text forms the text information and is input into the feature word recognition model. According to the input text information, what the feature word recognition model outputs is the classification probability of the current speech data block for the feature word. For example, the input text information is "Hello, phone", and the wake-up word is "Hello, TV". Among them, there are 4 speech data modules, and the feature word recognizes 3, namely "Hello, elec", then the output classification probability is 0.75. At this time, the recognition probability threshold is set to 0.7. It can be seen that the classification probability is greater than the recognition probability threshold, and it can be determined that the feature word is recognized in the speech data block. If the classification probability is equal to the recognition probability threshold, it can also be determined that the feature word is recognized in the speech data block.

[0085] In some embodiments, the wake-up word can be directly identified in the voice data stream. If the wake-up word is identified in the voice data stream, the control device 100 directly sends the wake-up word to the terminal device 200. For example, "Hello TV" is the wake-up word. When the user says "Hello TV, play music," the wake-up word "Hello TV" can be identified in the voice data stream, and the control device 100 can directly send the voice data stream of "Hello TV" to the terminal device 200. The terminal device 200 needs to perform a secondary verification on the wake-up word in the voice data stream to determine whether it is the real wake-up word "Hello TV." If it is determined to be the real wake-up word, the terminal device 200 can directly activate the voice interaction function based on the wake-up word. Alternatively, the terminal device 200 may not need to perform a secondary verification on the wake-up word in the voice data stream; that is, when the voice data stream of "Hello TV" is sent to the terminal device 200, the terminal device 200 directly activates the voice wake-up function based on the wake-up word "Hello TV."

[0086] In some embodiments, this application also provides a terminal device 200 for receiving a voice data stream sent by a control device 100. The voice data stream includes multiple voice data blocks. The voice data stream is streaming media data sent by the control device 100 when it identifies a feature word in the voice data block. The feature word includes at least one keyword in the voice interaction wake-up word. For example, the voice interaction wake-up word is "Friend, come out," and the voice data stream emitted by the user to the control device 100 is "Hello, friend." The feature word "friend" can be identified from the "Hello, friend" voice data stream, which is two characters in the voice interaction wake-up word. The terminal device 200 will receive the "Hello, friend" voice data stream. It then performs a secondary verification by identifying the wake-up word in the voice data stream. If the identified wake-up word is not "Friend, come out," the terminal device 200 will not respond to the voice interaction wake-up command in the voice data stream. If it is "Friend, come out," the terminal device 200 will respond to the voice interaction wake-up command in the voice data stream and activate the voice interaction function.

[0087] In some embodiments, the terminal device 200 acquires a voice data stream with a recognition success flag sent by the control device 100. The recognition success flag includes a first feature word. The terminal device 200 extracts the first feature word from the recognition success flag and recognizes a second feature word in the voice data stream. The input time of the voice corresponding to the first feature word is earlier than the input time of the voice corresponding to the second feature word; that is, the terminal device 200 receives the voice data stream of the first feature word first and then receives the voice data stream of the second feature word.

[0088] Among them, the voice features corresponding to the first feature word and the second feature word can be exactly the same or completely different. The first feature word and the second feature word are combined to generate a feature word, thereby awakening the voice interaction function of the terminal device 200. For example, the first feature word carried by the recognition success flag is "Hello", and the second feature word recognized by the terminal device 200 from the voice data stream is "Xiaoju". Combining the first feature word and the second feature word is the feature word "Hello Xiaoju" that can awaken the voice interaction function. Then the first feature word can also be "Xiaohai", and the second feature word is also "Xiaohai". Combining them is the feature word "Xiaohai Xiaohai".

[0089] In some embodiments, as Figure 11 shown, it is a schematic diagram of calculating the wake-up time using the existing wake-up process. When the control device 100 detects that the user says the complete wake-up word "Friend, come out" at a normal speaking speed, it takes 2000 milliseconds. Then, the wake-up word recognition module recognizes the wake-up word "Friend, come out", and this process takes 100 milliseconds. Then, the audio of the wake-up word "Friend, come out" is packaged and wirelessly transmitted to the terminal device 200. The audio packaging and wireless transmission process takes a total of 2800 milliseconds. The terminal device 200 receives the audio data sent by the control device 100 and decodes the audio data, and this process takes 200 milliseconds. Then, the terminal device 200 performs a secondary verification, that is, wakes up and recognizes the wake-up word again to determine whether it is really the wake-up word "Friend, come out", and this process takes 100 milliseconds. Therefore, from the user's complete utterance of the wake-up word "Friend, come out" to the terminal device 200's secondary verification to determine whether it is the real wake-up word "Friend, come out", and finally display the user interface (which takes 200 milliseconds). In summary, from the user's start of uttering the complete wake-up word "Friend, come out" until the user interface is displayed, it takes a total of 5400 milliseconds, that is, 5.4 seconds.

[0090] By analyzing the time calculation of the above entire process, the audio transmission time requires 2800 milliseconds, which occupies a large amount of time. Therefore, the present application optimizes the audio transmission between the control device 100 and the terminal device 200, that is, the control device 100 can start transmitting audio data to the terminal device 200 without the user uttering the complete wake-up word, which can greatly shorten the audio transmission time, and thus make the entire wake-up response process completed more quickly.

[0091] In some embodiments, as Figure 12The diagram illustrates the wake-up time calculation process according to an embodiment of this application. When the control device 100 detects that the user has just spoken "friend" at a normal speaking speed, 1000 milliseconds have elapsed. Then, the feature word recognition model, trained in pairs using the two-word approach of this application, identifies the feature word "friend," a process that takes 50 milliseconds. Simultaneously, the control device 100 begins wirelessly transmitting the voice data stream of "friend" to the terminal device 200. At this time, the user speaks "come out" at a normal speaking speed, taking 1000 milliseconds. While the voice transmission is ongoing, the feature word recognition model continues to identify the feature word "come out," thus simultaneously transmitting the voice data stream and recognizing the feature word. The process of uttering the wake word takes 950 milliseconds. Terminal device 200 receives and decodes the voice data stream, a process that takes 200 milliseconds. Then, terminal device 200 performs a second verification, again identifying the wake word "friend, come out," which takes 100 milliseconds. Therefore, from the user uttering the wake words "friend" and "come out" to terminal device 200 performing the second verification to determine if it's the genuine wake word "friend, come out," and finally displaying the user interface (taking 200 milliseconds), the total time from the user uttering the wake words "friend" and "come out" until the user interface is displayed is 3500 milliseconds, or 3.5 seconds. (Comparison...) Figure 11 and Figure 12 The wake-up time in the data shows that the wake-up time required to wake up using the wake-up process provided in this application is 1.9 seconds shorter than the wake-up time required to wake up using the existing wake-up process. In other words, the overall wake-up speed is improved by 1.9 seconds when using the wake-up process of this application.

[0092] In another embodiment of this application, the wake-up phrase "friends come out" is split into four keywords for separate wake-up recognition. When the user utters "friend" (500 milliseconds), the control device 100 begins wake-up recognition of "friend" and sends the voice data stream of "friends come out" to the terminal device 200 (850 milliseconds). This allows the terminal device 200 to recognize the other three keywords "friend," "come out," and "come" in the voice data stream, a process that takes 1600 milliseconds. Therefore, from the user uttering the wake-up phrases "friends" and "friends come out" to the terminal device 200 performing a secondary verification to determine if it is the genuine wake-up phrase "friends come out," and finally displaying the user interface (200 milliseconds), the total time from the user uttering "friends" and "friends come out" to the display of the user interface is 3150 milliseconds, or 3.15 seconds. This further improves the wake-up speed.

[0093] Some embodiments of this application achieve rapid response of the control device 100 and the terminal device 200, synchronizing wake-up recognition and audio transmission, thereby shortening wake-up time, improving wake-up performance, and providing users with a better experience.

[0094] Some embodiments of this application also provide an audio transmission method, the method comprising:

[0095] The control device 100 receives the voice data stream input by the user.

[0096] The control device 100 responds to a voice data stream, the voice data stream including multiple data blocks; and identifies feature words in the voice data blocks, the feature words including at least one keyword in a voice interaction wake-up word; the feature words include a first feature word and a second feature word; the input time of the voice corresponding to the first feature word precedes the input time of the voice corresponding to the second feature word.

[0097] If the first feature word is identified in the speech data block, a recognition success flag is set for the speech data stream, and the recognition success flag includes the first feature word.

[0098] The voice data stream with a recognition success flag is sent to the terminal device 200 so that the terminal device 200 can recognize the second feature word in the voice data stream.

[0099] In the above method embodiment, the control device 100 transmits voice data streams to the terminal device 200 while simultaneously recognizing them, thus shortening the time the user waits for the wake-up response and improving the user experience.

[0100] In summary, this application provides an audio transmission method, a control device 100, and a terminal device 200. By receiving a user-input voice data stream, which includes multiple voice data blocks, feature words are identified within the voice data blocks. These feature words include a first feature word and a second feature word. The input time of the voice corresponding to the first feature word precedes the input time of the voice corresponding to the second feature word. If the first feature word is identified in a voice data block, a recognition success flag is set for the voice data stream, and the voice data stream with the recognition success flag is sent to the terminal device, enabling the terminal device to identify the second feature word within the voice data stream. This application achieves rapid response from the control device 100 and the terminal device 200, shortening wake-up time and improving wake-up performance. This application not only efficiently and quickly completes audio transmission and secondary verification by the terminal device 200 but also enhances the user experience.

[0101] The same or similar parts among the various embodiments in this specification can be referred to mutually, and will not be repeated here.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0103] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the embodiments and various different variations of embodiments suitable for specific application considerations.

Claims

1. A control device, characterized in that, The control device includes: The communication module is configured to communicate with the terminal device. The control module is configured as follows: When receiving a voice data stream containing multiple voice data blocks from user input, feature words are identified in the voice data blocks. The feature words include a first feature word and a second feature word. The input time of the voice corresponding to the first feature word is earlier than the input time of the voice corresponding to the second feature word. The first feature word includes at least one keyword in the voice interaction wake-up word, and the second feature word includes the remaining keywords other than the first feature word. The speech data block is recognized based on the keyword corresponding to the first feature word; If the first feature word is identified in the speech data block, a recognition success flag is set for the speech data block corresponding to the first feature word, and the recognition success flag includes the first feature word; Send the voice data block with the recognition success flag to the terminal device; The remaining speech data blocks, excluding the speech data block corresponding to the first feature word, are identified based on the keyword corresponding to the second feature word. If the keyword corresponding to the second feature word is identified in the remaining voice data block, the voice data block corresponding to the second feature word is sent to the terminal device so that the terminal device can activate the voice interaction function when the wake-up recognition of the second feature word is successful.

2. The control device according to claim 1, characterized in that, The control module is also configured to: The wake word is broken down into multiple keywords; Identify the first keyword in the wake word within the voice data block; If the first keyword is identified in the voice data block, the voice data stream is sent to the terminal device so that the terminal device can identify the remaining keywords in the wake word other than the first keyword in the voice data stream.

3. The control device according to claim 1, characterized in that, The control module is also configured to: In the step of identifying feature words in the speech data block, a feature word recognition model is invoked, which is a neural network model trained based on speech sample data. The speech data block is converted into text information; Input the text information into the feature word recognition model; Obtain the classification probability of the current speech data block for the feature word output by the feature word recognition model; If the classification probability is greater than or equal to the recognition probability threshold, the feature word is determined to be recognized in the speech data block.

4. The control device according to claim 1, characterized in that, The control module is also configured to: Identify the wake word in the voice data stream; If the wake-up word is identified in the voice data stream, the wake-up word is sent to the terminal device so that the terminal device can activate the voice interaction function based on the wake-up word.

5. The control device according to claim 1, characterized in that, The control module is further configured to: In the step of identifying feature words in the speech data block, the feature words are identified sequentially in multiple speech data blocks according to time order; If the feature word is identified in any of the speech data blocks, the speech data block containing the identified feature word, as well as the remaining speech data block after the speech data block containing the identified feature word, are sent to the terminal device. If the feature word is not identified in any of the aforementioned voice data blocks, the voice data stream is not sent to the terminal device.

6. The control device according to claim 1, characterized in that, The control module is further configured to: In the step of responding to the voice data stream, an audio signal is acquired, the audio signal being generated via streaming radio. The audio signal is extracted at preset time intervals to generate multiple audio data blocks; Multiple audio data blocks are combined in chronological order to obtain the audio data stream.

7. A terminal device, characterized in that, The terminal device includes: The communicator is configured to communicate with the control equipment. The controller is configured as follows: Acquire a voice data stream, the voice data stream comprising multiple voice data blocks; the voice data stream is streaming media data sent by the terminal device when it identifies a keyword corresponding to a first feature word in the voice data blocks, the voice data stream carrying a recognition success flag, the recognition success flag including the first feature word; the first feature word includes at least one keyword in a voice interaction wake-up word; The system receives a voice data block corresponding to the second feature word sent by the control device when the second feature word is identified as a keyword in the remaining voice data block; the second feature word includes the remaining keywords other than the first feature word; the input time of the voice corresponding to the first feature word is earlier than the input time of the voice corresponding to the second feature word; Perform wake-up recognition on the speech data block corresponding to the second feature word; When wake-up recognition is successful, the voice interaction function is activated.

8. The terminal device according to claim 7, characterized in that, The controller is also configured to: Acquire the voice data stream; If the voice data stream identifies the second feature word, then a voice interaction command is identified in the voice data stream; Respond to voice interaction commands in the voice data stream.

9. The terminal device according to claim 7, characterized in that, The controller is also configured to: Acquire the voice data stream with a recognition success flag sent by the control device, wherein the recognition success flag includes the first feature word; Extract the first feature word from the recognition success marker; Identify the second feature word in the speech data stream; The first feature word and the second feature word are combined to generate the feature word.

10. An audio transmission method, characterized in that, The method includes: The control device receives a voice data stream containing multiple data blocks from user input, identifies feature words in the voice data blocks, and the feature words include at least one keyword in the voice interaction wake-up word; the feature words include a first feature word and a second feature word; the input time of the voice corresponding to the first feature word is earlier than the input time of the voice corresponding to the second feature word; The first feature word includes at least one keyword in the voice interaction wake-up word, and the second feature word includes the remaining keywords other than the first feature word; The speech data block is recognized based on the keyword corresponding to the first feature word; If the first feature word is identified in the speech data block, a recognition success flag is set for the speech data block corresponding to the first feature word, and the recognition success flag includes the first feature word; Send the voice data block with the recognition success flag to the terminal device; Recognition is performed on the remaining speech data blocks other than the speech data block corresponding to the first feature word based on the keyword corresponding to the second feature word; If the keyword corresponding to the second feature word is identified in the remaining voice data block, the voice data block corresponding to the second feature word is sent to the terminal device so that the terminal device can activate the voice interaction function when the wake-up recognition of the second feature word is successful.

Images