Control method, intelligent terminal and readable storage medium

Abstract

The present application proposes a control method, an intelligent terminal and a readable storage medium. The method comprises: collecting an audio signal of the current environment, detecting whether the audio signal contains a preset signal; determining the moment containing the preset signal as a target moment, and performing speech recognition on the audio signal cached at the previous moment based on the target moment; and executing a control instruction corresponding to the speech recognition result. The present application can reduce the number of human-computer interactions, thereby improving the efficiency of device control.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a control method, an intelligent terminal and a readable storage medium. Background Art

[0002] Nowadays, in order to improve the convenience of users using smart terminals such as mobile phones, notebooks and PDAs, many smart terminals have voice recognition capabilities. Voice assistants are usually set up and configured with corresponding wake-up words (this application can record it as the main wake-up word) to wake up the voice assistant of the smart terminal. For example, for speaker A, the user can say "Hello A" to start speaker A and wake up the voice recognition engine of speaker A (such as voice assistant). At this time, the user says "Play xx's song", the voice assistant can automatically recognize the voice control instructions contained in the audio signal, and automatically control speaker A to play xx's song by executing the voice control instructions. No manual operation is required by the user, which is very convenient.

[0003] In the process of conceiving and implementing this application, the inventors found that there are at least the following problems: if a user wants to use the voice control function of a smart terminal, he must first say the main wake-up word to wake up the voice assistant of the smart terminal, which is troublesome to use in certain application scenarios of the smart terminal.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the invention

[0005] In response to the above technical problems, the present application provides a control method, an intelligent terminal and a readable storage medium, which can reduce the number of human-computer interactions and thus improve the efficiency of device control.

[0006] In order to solve the above technical problems, the present application provides a control method, comprising the following steps:

[0007] S10: collecting an audio signal of the current environment, and detecting whether the audio signal contains a preset signal;

[0008] S20: Determine the moment containing the preset signal as the target moment, and perform speech recognition on the audio signal buffered at the previous moment based on the target moment;

[0009] S30: Execute the control instruction corresponding to the voice recognition result.

[0010] Optionally, the step S20 may include:

[0011] Determine or identify the wake-up mode corresponding to the current moment;

[0012] and detecting whether the audio signal includes a preset signal according to the wake-up mode.

[0013] Optionally, the detecting, according to the wake-up mode, whether the audio signal includes a preset signal includes:

[0014] The wake-up mode is a first wake-up mode, a target signal is selected from a preset signal set, and it is detected whether the audio signal contains a signal identical to the target signal; and / or,

[0015] The wake-up mode is a second wake-up mode, and detects whether the audio signal includes a signal similar to a signal in the preset signal set.

[0016] Optionally, the detecting, according to the wake-up mode, whether the audio signal includes a preset signal includes:

[0017] Determine or identify an audio duration corresponding to the audio signal, where the audio duration includes a start detection time, a first detection time, a second detection time, and an end detection time;

[0018] From the start detection time to the first detection time, detecting whether the audio signal contains a preset signal according to the wake-up mode; and / or,

[0019] From the second detection time to the termination detection time, detecting whether the audio signal includes a preset signal according to the wake-up mode.

[0020] Optionally, the control method further includes:

[0021] Detect the time interval corresponding to the current moment;

[0022] The wake-up value corresponding to the current moment is adjusted to the wake-up threshold corresponding to the time interval.

[0023] Optionally, before the step S20, the following steps are included:

[0024] Performing voiceprint recognition on the audio signal;

[0025] The voiceprint recognition result indicates that the source corresponding to the audio signal is the target source, and speech recognition is performed on the audio signal buffered at the previous moment.

[0026] Optionally, the voiceprint recognition result indicates that the source corresponding to the audio signal is a target source, and voice recognition is performed on the audio signal buffered at a previous moment, including:

[0027] The voiceprint recognition result indicates that the source corresponding to the audio signal is a target source, and a source type corresponding to the target source is determined or output;

[0028] Determining or identifying a speech recognition interval including the target time;

[0029] Based on the determined or output source type, speech recognition is performed on the buffered audio signal within the speech recognition interval.

[0030] Optionally, the performing speech recognition on the buffered audio signal within the speech recognition interval based on the determined or output source type includes:

[0031] The source type is a restricted source type, obtaining a preset voice library, wherein the preset voice library includes at least one reference text;

[0032] Converting the buffered audio signal into an audio text, and calculating or determining a similarity between the audio text and at least one reference text;

[0033] The semantics corresponding to the reference text with the greatest similarity is determined as the speech recognition result of the cached audio signal.

[0034] The present application also provides an intelligent terminal, comprising: a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the above method are implemented.

[0035] The present application also provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0036] The present application also provides a computer program product, comprising a computer program, wherein the computer program implements the steps of the above method when executed by a processor.

[0037] As described above, the control method of the present application includes the steps of: collecting the audio signal of the current environment, and detecting whether the audio signal contains a preset signal, then determining the moment containing the preset signal as the target moment, taking the target moment as the reference, performing voice recognition on the audio signal cached at the previous moment, and finally, executing the control instruction corresponding to the voice recognition result. The control method of the present application, when detecting that the audio signal contains a preset signal, determines the moment containing the preset signal as the target moment, and taking the target moment as the reference, performs voice recognition on the audio signal cached at the previous moment. Through the above technical solution, when the user needs to control the device by voice, there is no need to perform the functions of voice wake-up and voice recognition respectively through two audio signals, thereby reducing the number of human-computer interactions and improving the efficiency of device control. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0039] Figure 1 A schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application;

[0040] Figure 2 A communication network system architecture diagram provided for an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of a scenario of a control method provided in an embodiment of the present application;

[0042] Figure 4 It is a flow chart of the control method provided in the embodiment of the present application;

[0043] Figure 5 It is a schematic diagram of the time point division of the audio duration provided by an embodiment of the present application;

[0044] Figure 6 This is a schematic diagram of an interface for displaying prompt information on a mobile phone screen provided in an embodiment of the present application;

[0045] Figure 7 This is a schematic diagram of an interface for displaying prompt information on a display screen of a tablet computer provided in an embodiment of the present application;

[0046] Figure 8 is a schematic diagram of a process for controlling a video application provided by an embodiment of the present application;

[0047] Fig. 9 is a schematic diagram of a process of controlling a social application provided by an embodiment of the present application;

[0048] Fig.10 is a schematic diagram of a process of controlling a camera provided in an embodiment of the present application;

[0049] Fig.11 is another flow chart of the control method provided in the embodiment of the present application;

[0050] Fig.12 is another scenario schematic diagram of the control method provided in an embodiment of the present application;

[0051] Fig.13 is a schematic diagram of the structure of the device control device provided in an embodiment of the present application;

[0052] Fig.14 It is another structural schematic diagram of the equipment control device provided in an embodiment of the present application.

[0053] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0054] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0055] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0056] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate that there are the described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or", "and / or", "including at least one of the following" etc. used in this application can be interpreted as inclusive, or mean any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”, and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will only occur when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0057] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0058] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0059] It should be noted that in this article, step codes such as S10 and S20 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence. When implementing the step, those skilled in the art may execute S20 first and then S10, etc., but these should all be within the scope of protection of this application.

[0060] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0062] The smart terminal may be implemented in various forms. For example, the smart terminal described in this application may include mobile terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0063] The following description will be made by taking a mobile terminal as an example, and those skilled in the art will understand that, in addition to components specifically used for mobile purposes, the construction according to the embodiments of the present application can also be applied to fixed-type terminals.

[0064] See also Figure 1 , which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application, the mobile terminal 100 may include: RF (Radio Frequency, radio frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0065] Combine the following Figure 1 The following is a detailed introduction to the various components of the mobile terminal:

[0066] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communications can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution) and TDD-LTE (Time Division Duplexing-Long Term Evolution) and 5G, etc.

[0067] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not a necessary component of the mobile terminal and can be omitted as needed without changing the essence of the application.

[0068] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0069] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and other operating modes, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0070] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured on the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0071] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0072] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute the command sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0073] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Then, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.

[0074] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0075] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area. Optionally, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0076] The processor 110 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, so as to monitor the mobile terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0077] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0078] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0079] In order to facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0080] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present application, wherein the communication network system is an LTE system of universal mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence for communication.

[0081] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0082] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide UE 201 with access to EPC 203 .

[0083] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, and provides bearer and connection management. HSS 2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and store some user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0084] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.

[0085] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA and future new network systems (such as 5G), etc., without limitation here.

[0086] Based on the above-mentioned mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0087] See also Figure 3 , Figure 3 It is a scene diagram of the control method provided by the embodiment of the present application. Taking the device control device integrated in the mobile phone as an example for specific explanation, in some embodiments, when the user needs to check the weather forecast through the mobile phone, he can issue a voice command "What is the weather like tomorrow, Xiaoyin", and the mobile phone collects the audio signal of the current environment in standby mode. Then, the mobile phone uses the current moment as a reference to detect whether the audio signal contains a wake-up word within a preset time range. Then, when the mobile phone detects that the audio signal contains a wake-up word within the preset time range, it caches the audio signal and performs voiceprint recognition on the audio signal. For example, in this embodiment, "Xiaoyin" is the wake-up word. The mobile phone uses the current moment as a reference to detect whether the audio signal contains a wake-up word before T seconds at the current moment and after T seconds at the current moment. When the mobile phone detects that the audio signal contains a wake-up word within the preset time range, it caches the audio signal and performs voiceprint recognition on the audio signal. Finally, the mobile phone executes the control instruction corresponding to the voice recognition result.

[0088] The control method of the present application sequentially performs wake-up word detection, voiceprint recognition, and speech recognition on the same audio signal. When the user needs to control the device by voice, there is no need to perform voice wake-up and speech recognition separately through two audio signals. Thus, the number of human-computer interactions is reduced, thereby improving the efficiency of device control.

[0089] The following is a detailed description of the technical solution shown in the present application through specific embodiments. Taking the mobile phone control solution as an example, it is also applicable to control smart homes, tablet computers, and personal computers. It should be noted that the following embodiments can exist alone or in combination with each other. For the same or similar content, it will not be repeated in different embodiments.

[0090] See also Figure 4 , Figure 4 : is a flow chart of a control method provided in an embodiment of the present application. The control method may specifically include:

[0091] S10: Collect the audio signal of the current environment and detect whether the audio signal contains a preset signal.

[0092] Optionally, in some embodiments, the audio signal of the current environment can be collected when the device is in standby mode, which refers to the state where a smart terminal such as a mobile phone or a computer is turned on but does not perform any substantial work (i.e., does not perform various operations on files and programs). Standby mode is to save the data currently in operation in the memory, and the machine only supplies power to the memory, while components such as the hard disk, screen, and CPU are powered off. In an embodiment of the present application, the standby mode may include an airplane mode, a charging mode, and a silent mode, etc. In the standby mode, the surrounding sounds are continuously monitored, and effective voice information is extracted from the monitored sounds. That is, in an embodiment of the present application, the audio signal may be an audio signal emitted by a person, and the preset signal may be a wake-up word, a piece of music, or a collision sound, etc. The wake-up word may be a word, a single word, a letter, a number, or a character, etc. pre-set by the user, and the preset signal is used as the wake-up word. Optionally, in some embodiments, it is possible to continuously detect whether the audio signal contains a wake-up word, such as using a wake-up model to determine the probability of the audio signal belonging to a preset wake-up word, and score the reference frequency in the audio signal according to the probability. Then, according to the score value, a score distribution of the wake-up word and the non-wake-up word is constructed, and a ROC curve (receiver operating characteristic curve) is generated based on the score distribution. Finally, the EER (Equal Error Rate) point on the ROC curve can be selected and the point is set as the silent wake-up threshold.

[0093] The wake-up model needs to be trained before use. To train the wake-up model, you first need to define the wake-up word. Generally, a word of 3-4 syllables is defined as the wake-up word, for example, "Xiao Ai classmate", "Hello Xiaoda" and "Xiao Zhang classmate", etc. After that, the pronunciation of the wake-up word needs to be collected. For example, the pronunciation of the wake-up word can be collected according to the rule of thousands of people and thousands of hours. Finally, the collected data is input into the prepared model to train the model. The basic algorithm model can adopt keyword spotting (KWS) based on template matching, KWS based on Markov model, and KWS based on deep learning. In actual use, the number of wake-up words and the specific content of the wake-up words can be adjusted according to actual needs. When it is detected that the audio signal contains a preset signal, step S20 is executed.

[0094] Still taking the preset signal as a wake-up word as an example for explanation, within the time range of S seconds before the current moment to S seconds after the current moment, detecting whether the audio signal contains the wake-up word, wherein S seconds can be set according to the length of the wake-up word. Optionally, in some embodiments of the present application, the wake-up mode corresponding to the current moment can be determined, and then, within the preset time range, the audio signal is detected according to the wake-up mode. Whether the wake-up word is included, that is, optionally, in some embodiments, step S20 may specifically include:

[0095] (11) determining or identifying the wake-up mode corresponding to the current moment;

[0096] (12) Detect whether the audio signal contains a wake-up word based on the wake-up pattern.

[0097] Optionally, the wake-up modes corresponding to different times are different, which can be set by the user through the device, or by the engineering staff before the device leaves the factory. Of course, the user can also add a different wake-up mode based on the already set wake-up mode.

[0098] In order to meet the usage requirements in different scenarios, the requirements for wake-up are different, and the application scenarios may include home scenarios, meeting scenarios, and public scenarios. Optionally, in a meeting scenario, in order to facilitate users to quickly wake up the device and control it in a meeting scenario, the control method of the present application may be used to control the target device, and in order to avoid accidentally waking up the device during a meeting, resulting in the interruption of the meeting, in some embodiments, it may be detected whether the audio signal contains the same wake-up word as the target signal. Optionally, in a home scenario, in order to improve the efficiency of user control of the device, the difficulty of waking up the device may be reduced, that is, detecting whether the audio signal contains a wake-up word similar to that in a preset signal set. For example, when a user needs to wake up the air conditioner, the preset signal set only contains the wake-up word "air conditioner". When the voice emitted by the user contains "cold air" or "air conditioning", the air conditioner is awakened. That is, optionally, in some embodiments, the step of "detecting whether the audio signal contains a wake-up word according to the wake-up mode within a preset time range" may specifically include:

[0099] (21) the wake-up mode is the first wake-up mode, a target signal is selected from a preset signal set, and the audio signal is detected to see whether it contains the same wake-up word as the target signal; and / or,

[0100] (22) If the wake-up mode is the second wake-up mode, detect whether the audio signal includes a signal similar to a signal in the preset signal set.

[0101] Since most users have the voice wake-up habit of wake-up word + instruction, or instruction + wake-up word, it can be seen that the wake-up word is often located in the first few seconds or the last few seconds of the audio signal. Therefore, in order to improve the efficiency of device wake-up and thus achieve the technical effect of improving the efficiency of device control, optionally, in some embodiments, it is also possible to detect whether there is a wake-up word in a certain section of the audio signal, that is, the step of "detecting whether the audio signal contains a wake-up word according to the wake-up mode" can specifically include:

[0102] (31) determining or identifying the audio duration corresponding to the audio signal;

[0103] (32) from the start detection time to the first detection time, detecting whether the audio signal contains a preset signal according to the wake-up mode; and / or,

[0104] (33) From the second detection moment to the end detection moment, detect whether the audio signal contains a preset signal according to the wake-up mode.

[0105] Optionally, the audio duration includes a start detection time c, a first detection time t1, a second detection time t2, and an end detection time e, wherein the first detection time t1 is later than the start detection time c, the second detection time t2 is later than the first detection time t1, and the end detection time e is later than the second detection time t2. Figure 5 As shown, for example, the audio duration of the audio signal is 30 seconds, the start detection time c is the 0th second of the audio signal, the first detection time t1 is the 1st second of the audio signal, the second detection time t2 is the 29th second of the audio signal, and the end detection time e is the 30th second of the audio signal. Of course, the time interval between the start detection time c and the first detection time t1 can be 2 seconds, 3 seconds, 5 seconds, etc., and the time interval between the second detection time t2 and the end detection time e can be 2 seconds, 3 seconds, 5 seconds, etc., which can be set according to actual conditions. It can be understood that the time interval between the start detection time c and the first detection time t1 can be the same as or different from the time interval between the second detection time t2 and the end detection time e.

[0106] For example, after determining that the audio duration corresponding to the audio signal is 50 seconds, the time interval between the first detection moment and the start detection moment, and the time interval between the second detection moment and the end detection moment can be determined according to the audio duration. In this way, the first detection moment and the second detection moment can be flexibly determined according to the audio duration. In some embodiments of the present application, between the start detection moment and the first detection moment, it is detected whether the audio signal contains the wake-up word according to the wake-up mode; when it is detected that the audio signal does not contain the wake-up word between the start detection moment and the first detection moment, then between the second detection moment and the end detection moment, it is detected whether the audio signal contains the wake-up word according to the wake-up mode; or, between the second detection moment and the end detection moment, it is detected whether the audio signal contains the wake-up word according to the wake-up mode; when it is detected that the audio signal does not contain the wake-up word between the second detection moment and the end detection moment, then between the start detection moment and the first detection moment, it is detected whether the audio signal contains the wake-up word according to the wake-up mode; of course, it is also possible to simultaneously detect whether the audio signal contains the wake-up word between the start detection moment and the first detection moment, and between the second detection moment and the end detection moment, wherein the method of detecting whether the audio signal contains the wake-up word according to the wake-up mode can refer to the previous embodiment and will not be repeated here.

[0107] It should be noted that in the simultaneous detection scheme, when the start detection moment and the first detection moment correspond to the first wake-up mode, and the second detection moment and the end detection moment correspond to the second wake-up mode, the first wake-up mode can be used to detect whether the audio signal contains the wake-up word, or the second wake-up mode can be used to detect whether the audio signal contains the wake-up word. Of course, it is also possible to detect whether the audio signal contains the wake-up word according to the first wake-up mode between the start detection moment and the first detection moment, and to detect whether the audio signal contains the wake-up word according to the second wake-up mode between the second detection moment and the end detection moment.

[0108] It should also be noted that in the simultaneous detection scheme, when the start detection moment corresponds to the first wake-up mode, and the first detection moment, the second detection moment, and the termination detection moment correspond to the second wake-up mode, the wake-up mode corresponding to the start detection moment is used as the reference, that is, between the start detection moment and the first detection moment, the first wake-up mode is used to detect whether the audio signal contains the wake-up word, and, between the second detection moment and the termination detection moment, the first wake-up mode is used to detect whether the audio signal contains the wake-up word. It is also possible to use the wake-up mode corresponding to the first detection moment, the second detection moment, and the termination detection moment as the reference, that is, between the start detection moment and the first detection moment, the second wake-up mode is used to detect whether the audio signal contains the wake-up word, and, between the second detection moment and the termination detection moment, the second wake-up mode is used to detect whether the audio signal contains the wake-up word.

[0109] Optionally, in some embodiments, the average value between the start detection time, the first detection time, the second detection time, and the end detection time can also be calculated to obtain a reference detection time, and the audio signal is detected whether the wake-up word is included according to the wake-up mode corresponding to the reference detection time. For example, the start detection time, the first detection time, the second detection time, and the end detection time correspond to the 1st second, the 6th second, the 18th second, and the 23rd second of the audio duration, respectively, and the corresponding average value is 12, that is, the reference detection time is the 12th second. If the wake-up mode corresponding to the 12th second is the first wake-up mode, then the audio signal is detected whether the wake-up word is included according to the first wake-up mode, and so on.

[0110] In addition, it should be noted that, in some embodiments, in the standby mode, in order to reduce the probability of false awakening of the user, the corresponding wake-up value may be adjusted according to the time interval corresponding to the current moment, that is, optionally, in some embodiments, it may also specifically include:

[0111] (41) Detecting the time interval corresponding to the current moment;

[0112] (42) The wake-up value corresponding to the current moment is adjusted to the wake-up threshold corresponding to the time interval.

[0113] In an embodiment of the present application, the moment of collecting the audio signal of the current environment is determined as the current moment, that is, in standby mode, each moment of collecting the audio signal is the current moment, that is, in the device control scheme of the present application, the wake-up value can be adjusted according to different current moments. For example, the time interval from 8 am to 10 pm can be set as the first time interval, and the time interval from 10 pm to 8 am can be set as the second time interval, and the wake-up threshold of the first time interval can be set to 40, and the wake-up advance of the second time interval can be set to 10, so as to reduce the probability of the user waking up the device by mistake.

[0114] When it is detected that the audio signal contains the wake-up word, step S20 is executed; when it is detected that the audio signal does not contain the wake-up word, the process returns to the step of collecting the audio signal of the current environment.

[0115] S20: Determine the moment containing the preset signal as the target moment, and perform speech recognition on the audio signal buffered at the previous moment based on the target moment.

[0116] For example, specifically, the audio text corresponding to the audio signal is "How's the weather today, Xiaoyin", where "Xiaoyin" is the wake-up word, and the moment corresponding to "Xiaoyin" is determined as the target moment. In some embodiments, if multiple identical wake-up words appear in the audio signal, such as "Xiaoyin, how's the weather, Xiaoyin", the moment of the first wake-up word can be determined as the target moment, or the moment of the last wake-up word can be determined as the target moment; of course, the moment of the first wake-up word can also be determined as the first target moment, and the moment of the last wake-up word can be determined as the second target moment, such as "Xiaoyin Xiaoyin, help me turn up the screen brightness, Xiaoyin, check the weather forecast again", and voice recognition can be performed based on the first target moment and the second target moment in the future.

[0117] Optionally, before step S20, voiceprint recognition can also be performed on the audio signal. For example, after the wake-up chip recognizes the wake-up word, the central processing unit can be started to perform voiceprint recognition on the audio signal. If the voiceprint recognition result meets the preset conditions, step S50 is executed; if the voiceprint recognition result does not meet the preset conditions, the standby mode continues to be entered, and the central processing unit enters the sleep state.

[0118] The so-called voiceprint is the sound wave spectrum that carries speech information displayed by electroacoustic instruments. The production of human language is a complex physiological and physical process between the human body's language center and the vocal organs. The vocal organs used by people when speaking - tongue, teeth, larynx, lungs, nasal cavity - vary greatly in size and shape from person to person, so the voiceprints of any two people are different.

[0119] Optionally, voiceprint recognition, as a method of biometric information identification, includes two stages: user registration and user identity identification. In the registration stage, the voice is mapped to a user model through a series of processes. In the identification stage, a voice with an unknown identity is matched with the model for similarity, and then the identity of the unknown voice is judged to be consistent with the identity of the registered voice.

[0120] Optionally, during the voiceprint registration stage, the registered user can read a registration string aloud. For example, the registration string is a digital string 123823218, which includes four numbers "1", "2", "3", and "8". The voiceprint recognition device then extracts voiceprint features and trains voiceprint models based on the voice segments corresponding to each character in the registration voice information to obtain voiceprint features of the voice segments corresponding to "1", "2", "3", and "8". Furthermore, the voiceprint features of the voice segments corresponding to "1", "2", "3", and "8" are obtained by training the universal background model (UBM) corresponding to the preset corresponding characters to obtain feature vectors corresponding to each character in the registration voice information, including a feature vector corresponding to the number "1", a feature vector corresponding to the number "2", a feature vector corresponding to the number "3", and a feature vector corresponding to the number "8". Then, the voiceprint recognition device can save the feature vectors corresponding to multiple characters in the registration voice information read by different registered users during the voiceprint registration stage in the model library of the voiceprint recognition device.

[0121] In the identity recognition stage, the verification user, i.e., the user of unknown identity, reads a verification string. For example, the verification string is a digital string 56562561. Then, the voiceprint recognition device extracts voiceprint features and trains voiceprint models according to the voice segments corresponding to the characters in the verification voice information generated when the verification user reads aloud, and obtains GMMs corresponding to "1", "2", "5", and "6". Then, in combination with the preset UBMs corresponding to the corresponding characters, the feature vector of the verification voice information of the verification user can be calculated, including a feature vector corresponding to the number "0", a feature vector corresponding to the number "1", a feature vector corresponding to the number "5", and a feature vector corresponding to the number "8". Then, the similarity scores between the feature vectors corresponding to "1", "2", "5", and "6" in the verification voice information and the feature vectors corresponding to "1", "2", "5", and "6" in the registration voice information are calculated respectively. If the similarity score reaches the preset verification threshold, the verification user is determined as the registered user corresponding to the registration voice information. That is, optionally, in some embodiments, before step S20, the following may also be included:

[0122] (51) performing voiceprint recognition on audio signals;

[0123] (52) The voiceprint recognition result indicates that the source corresponding to the audio signal is the target source, and voice recognition is performed on the audio signal cached at the previous moment.

[0124] For example, specifically, the target moment can be used as a reference, and the audio signal from Q seconds before the target moment to Q seconds after the target moment can be determined in the cached audio signal to obtain the audio signal to be recognized, and then, speech recognition can be performed on the audio signal to be recognized, wherein the specific value of Q can be set according to actual conditions, and can be 5 seconds, 9 seconds, 10 seconds, and so on.

[0125] In some embodiments of the present application, voice recognition can be performed on the audio signal to be recognized according to different user types. For example, for user A, he can wake up all components, applications, and functions of the device. When performing the voice recognition process, the audio signal to be recognized can be converted into a corresponding voice text according to the voice features of the audio signal to be recognized, and the corresponding voice recognition result can be output. For user B, he can only wake up some components, applications, and / or functions of the device. For example, if user B is a child, he cannot wake up game applications, video applications, or query video information within a certain period of time. When performing the voice recognition process, when it is recognized that the voice text contains components, applications, and / or functions that are restricted from use, it is detected whether the target moment is within the corresponding usage time period. If so, the voice recognition result is output; if not, the corresponding prompt information is generated. For example, the prompt information can be displayed on the screen of the mobile phone, such as Figure 6 As shown, in Figure 6 In the corresponding embodiment, user B is a child, and he wakes up the video application outside the usage time period. Therefore, in this solution, "Please start the application within the allowed time period" is displayed on the display screen of the mobile phone. That is, optionally, in some embodiments, step S50 may specifically include:

[0126] (51) when the voiceprint recognition result indicates that the source corresponding to the audio signal is the target source, determining or outputting the source type corresponding to the target source;

[0127] (52) determining or identifying a speech recognition interval including a target time;

[0128] (53) Based on the determined or output source type, speech recognition is performed on the cached audio signal within the speech recognition interval.

[0129] Optionally, the target user is a user who has pre-entered voiceprint information in the device, the user type indicates the user's identity type, such as parent and child, or device holder and device visitor, etc., the voice recognition interval includes the voice recognition starting point and the voice recognition end point, and the target time is between the voice recognition starting point and the voice recognition end point.

[0130] Optionally, the user type may include a restricted user type and an unrestricted user type. As the name implies, a user corresponding to the restricted user type can only control some components and / or applications of the device through voice, while a user corresponding to the unrestricted user type can control all components and applications of the device through voice. For example, in a smart home scenario, Figure 7 As shown, the tablet computer can be used to control the turning on and off of all smart appliances in the house, wherein user A is an unrestricted user, that is, the user type corresponding to the tablet computer is an unrestricted user type, and user B is a restricted user, that is, the user type corresponding to user B is a restricted user type, and user B can only control the turning on or off of fans, televisions, and air conditioners. When user B issues a voice command: turn off the lights of the whole house, at this time, the tablet computer performs voiceprint recognition on user B and recognizes that the user type corresponding to user B is a restricted user type. Then, the tablet computer can detect the similarity between the voice text corresponding to the voice command of user B and the voice text of the preset voice library. It can be seen from the above that user B cannot control the turning off of the lights. Therefore, the tablet computer can display a corresponding prompt message such as "You cannot perform this operation" through its display screen, and can also broadcast a corresponding voice reminder through a speaker, that is, optionally, in some embodiments, the step of "based on the determined user type, performing voice recognition on the cached audio signal within the voice recognition interval" can specifically include:

[0131] (61) When the user type is a restricted user type, obtaining a preset voice library;

[0132] (62) converting the cached audio signal into an audio text, and calculating the similarity between the audio text and each reference text;

[0133] (63) The semantics corresponding to the reference text with the greatest similarity is determined as the speech recognition result of the cached audio signal.

[0134] Optionally, the preset voice library may include multiple reference texts. The reference texts in the preset voice library may be pre-entered by the manufacturer or by the user. Of course, a basic voice library may be pre-entered before the device leaves the factory, and the user may record new voices on the basic voice library to form a preset voice library.

[0135] S30: Execute a control instruction corresponding to the speech recognition result.

[0136] Among them, the corresponding application can be started, the application interface can be displayed and / or the device component of the target device can be called according to the speech recognition result, that is, optionally, in some embodiments, step S30 can specifically include:

[0137] (71) Start the target application;

[0138] (72) displaying an application interface; and / or,

[0139] (73) Call the device component of the target device.

[0140] For example, see Figure 8 In standby mode, when the user inputs the voice: Xiaoyin, open video application XX, after receiving the audio signal, the mobile phone detects whether the audio signal contains the wake-up word. When the mobile phone detects that the audio signal contains the wake-up word "Xiaoyin", the moment when the wake-up word is detected is determined as the target moment. Then, it enters the ready mode and performs voiceprint recognition on the audio signal. When the voiceprint recognition result indicates that the user corresponding to the audio signal is the target user, it enters the use mode. The mobile phone uses the target moment as a reference to perform voice recognition on the cached audio signal. Based on this, the mobile phone can recognize that the audio text corresponding to the audio signal sent by the user is: Open video application XX, and the mobile phone can start video application XX.

[0141] For example, see Fig. 9 In standby mode, when the user inputs the voice: Xiaoyin, the conversation page with contact K is opened. After receiving the audio signal, the mobile phone detects whether the audio signal contains the wake-up word. When the mobile phone detects that the audio signal contains the wake-up word "Xiaoyin", the moment when the wake-up word is detected is determined as the target moment. Then, it enters the ready mode and performs voiceprint recognition on the audio signal. When the voiceprint recognition result indicates that the user corresponding to the audio signal is the target user, it enters the use mode. The mobile phone performs voice recognition on the cached audio signal based on the target moment. Based on this, the mobile phone can recognize that the audio text corresponding to the audio signal sent by the user is: Open the search engine page, the mobile phone can start the social application, select contact K in the contact list of the social application, and then display the conversation page with contact K.

[0142] For example, see Fig.10 In standby mode, when the user inputs the voice: Xiaoyin, help me take a selfie, after the mobile phone receives the audio signal, the mobile phone detects whether the audio signal contains the wake-up word. When the mobile phone detects that the audio signal contains the wake-up word "Xiaoyin", the moment when the wake-up word is detected is determined as the target moment. Then, it enters the ready mode and performs voiceprint recognition on the audio signal. When the voiceprint recognition result indicates that the user corresponding to the audio signal is the target user, it enters the use mode. The mobile phone uses the target moment as a reference to perform voice recognition on the cached audio signal. Based on this, the mobile phone can recognize that the audio text corresponding to the audio signal sent by the user is: Help me take a selfie. The mobile phone can then start the front camera and display the corresponding camera page.

[0143] In addition, in the smart home scenario, the user can control the opening or closing of multiple devices through a master control device. The control method of the present application can control the opening and closing of multiple devices by waking up the master device. Specifically, the corresponding devices can be controlled to be turned on and / or turned off according to the user's voice control record. That is, optionally, in some embodiments, step S60 may specifically include:

[0144] (81) Obtain the user account currently logged in to the device;

[0145] (82) Collecting voice control records of user accounts;

[0146] (83) According to the voice control record, execute the control instruction corresponding to the voice recognition result

[0147] For example, the user account currently logged in to the main control device is user account S. When the main control device receives the voice from the user: Xiaoyin, turn on all working devices (i.e., receive audio signals), the main control device performs wake-up word detection, voiceprint detection and voice recognition on the audio signal. The main control device can obtain the voice control record of user account S, and the voice control record indicates that the working devices turned on by the user in the past month are the first device A1 and the second device A2. Subsequently, the main control device turns on the first device A1 and the second device A2.

[0148] It should be noted that when the main control device turns on the first device A1 and the second device A2, the third device A3 is in the turned-on state. At this time, the main control device can turn off the third device A3, or ask the user through voice broadcast. When the main control device receives the user's voice response of "yes", the third device A3 is turned off. If the main control device does not receive the user's voice response of "yes" within the preset time, the third device A3 remains turned on.

[0149] As can be seen from the above, the control method provided by the present application, in standby mode, collects the audio signal of the current environment, and then detects whether the audio signal contains a wake-up word. When the audio signal is detected to contain a wake-up word, the moment when the wake-up word is detected is determined as the target moment. Next, the audio signal is subjected to voiceprint recognition. When the voiceprint recognition result indicates that the user corresponding to the audio signal is the target user, the target moment is used as a reference to perform speech recognition on the cached audio signal. Finally, the control instruction corresponding to the speech recognition result is executed. It can be seen that the control method of the present application, when it is detected that the audio signal contains a preset signal, determines the moment containing the preset signal as the target moment, and uses the target moment as a reference to perform speech recognition on the audio signal cached at the previous moment. Through the above technical solution, when the user needs to control the device by voice, there is no need to respectively perform the functions of voice wake-up and voice recognition through two audio signals, thereby reducing the number of human-computer interactions and improving the efficiency of device control.

[0150] The present application also provides a control method, wherein the device control device is integrated in a mobile phone, see Fig.11 , Fig.11 This is another flow chart of the control method provided by this application. The specific process is as follows:

[0151] S21. The mobile phone collects audio signals of the current environment in the standby mode.

[0152] For example, when the mobile phone is in flight mode, the wake-up chip it is equipped with will continuously monitor the sounds of the surrounding environment and cache audio of a certain length. In this embodiment, the audio signal of the current environment is monitored by the wake-up chip to avoid frequent calls to the central processor that lead to increased power consumption. Therefore, in this embodiment, the power consumption of the mobile phone can be reduced by using the wake-up chip to collect the audio signal of the current environment.

[0153] S22: The mobile phone detects whether the audio signal contains a wake-up word.

[0154] For example, the mobile phone detects whether the audio signal contains a wake-up word within a time range from S seconds before the current moment to S seconds after the current moment, where S seconds can be set according to the length of the wake-up word. Optionally, the mobile phone can use a wake-up model to detect whether the audio signal contains a wake-up word. For example, the mobile phone uses the wake-up model to determine the probability of a preset wake-up word in the audio signal, and the mobile phone scores the reference frequency in the audio signal according to the probability. Then, the mobile phone constructs a score distribution of wake-up words and non-wake-up words according to the score value, and generates a ROC curve (receiveroperating characteristic curve) based on the score distribution. Finally, the mobile phone can also select an EER (Equal Error Rate) point on the ROC curve and set the point as the silent wake-up threshold.

[0155] S23. When the mobile phone detects that the audio signal contains a wake-up word, the moment when the wake-up word is detected is determined as the target moment.

[0156] Please refer to the embodiment of S30 for details, which will not be described in detail here.

[0157] S24. The mobile phone performs voiceprint recognition on the audio signal.

[0158] After the mobile phone detects the wake-up word and determines the moment when the wake-up word is detected as the target moment, the mobile phone can use a preset voiceprint recognition model to perform voiceprint recognition on the audio signal. For details, please refer to the embodiment of step S40, which will not be repeated here.

[0159] S25. When the voiceprint recognition result indicates that the user corresponding to the audio signal is the target user, the mobile phone performs voice recognition on the cached audio signal based on the target time.

[0160] For example, specifically, the mobile phone can use the target time as a reference, determine the audio signal from Q seconds before the target time to Q seconds after the target time in the cached audio signal, obtain the audio signal to be recognized, and then perform speech recognition on the audio signal to be recognized, where the specific value of Q can be set according to actual conditions, and can be 5 seconds, 9 seconds, 10 seconds, etc.

[0161] It should be noted that, in some embodiments, if the voiceprint recognition result indicates that the user corresponding to the audio signal is not the target user, the mobile phone can issue a security warning or start a retrieval function.

[0162] S26. The mobile phone executes a control instruction corresponding to the voice recognition result.

[0163] For example, when the user's voice is "Xiaoyin, I want to listen to XX", XX is the name of a song. In the scheme of the present application, the corresponding song can be played after the user's voiceprint recognition and voice recognition. The user does not need to wake up and wait for the voice assistant to reply before the user issues instructions to control the device to perform the corresponding operation. This reduces the number of interactions between the user and the device, thereby improving the efficiency of controlling the device.

[0164] From the above, it can be seen that in the control method provided by the present application, the mobile phone collects the audio signal of the current environment in standby mode, and then the mobile phone detects whether the audio signal contains the wake-up word. When the mobile phone detects that the audio signal contains the wake-up word, the moment when the wake-up word is detected is determined as the target moment. Then, the mobile phone performs voiceprint recognition on the audio signal. When the voiceprint recognition result indicates that the user corresponding to the audio signal is the target user, the mobile phone performs speech recognition on the cached audio signal based on the target moment, and finally executes the control instruction corresponding to the speech recognition result. It can be seen that the control method of the present application performs wake-up word detection, voiceprint recognition and speech recognition on the same audio signal in sequence. When the user needs to control the device by voice, there is no need to perform voice wake-up and speech recognition respectively through two audio signals. As a result, the number of human-computer interactions is reduced, thereby improving the efficiency of device control.

[0165] In order to facilitate further understanding of the control method of the present application, the present application also provides a device control system (hereinafter referred to as the control system), which is specifically described below using the smart home scene as an example. Fig.12 , Fig.12 This is another scenario schematic diagram of the control method provided by this application, which is as follows:

[0166] The control system includes a mobile phone 10, a first device to be controlled 20, a second device to be controlled 30, a third device to be controlled 40, a fourth device to be controlled 50 and a fifth device to be controlled 60, wherein the mobile phone 10 serves as a management device of the entire smart home system, and the mobile phone 10 establishes connections with the first device to be controlled 20, the second device to be controlled 30, the third device to be controlled 40, the fourth device to be controlled 50 and the fifth device to be controlled 60 through a wireless network, so that a user can control the first device to be controlled 20, the second device to be controlled 30, the third device to be controlled 40, the fourth device to be controlled 50 and the fifth device to be controlled 60 through the mobile phone 10.

[0167] Optionally, the first device to be controlled 20 and the second device to be controlled 30 are both smart speakers, the third device to be controlled 40 is a smart TV, the fourth device to be controlled 50 is a smart air conditioner, and the fifth device to be controlled 60 is a smart desk lamp. When the main control device (mobile phone 10) receives the voice from the user: Xiaoyin, enter the listening mode, at this time, the mobile phone 10 performs wake-up word detection, voiceprint detection and voice recognition on the audio signal, and based on the voice recognition result of the audio signal, determines that the devices associated with the "listening mode" are: the first device to be controlled 20 and the second device to be controlled 30. At this time, the mobile phone 10 turns on the first device to be controlled 20 and the second device to be controlled 30, and can also broadcast the inquiry information: Master, do you need to turn on the air conditioner? When the mobile phone 10 receives the user's response voice of "yes", the mobile phone 10 turns on the air conditioner, that is, turns on the fourth device to be controlled 50.

[0168] From the above, it can be seen that in the control method provided by the present application, the mobile phone collects the audio signal of the current environment in standby mode, and then the mobile phone detects whether the audio signal contains the wake-up word. When the mobile phone detects that the audio signal contains the wake-up word, the moment when the wake-up word is detected is determined as the target moment. Then, the mobile phone performs voiceprint recognition on the audio signal. When the voiceprint recognition result indicates that the user corresponding to the audio signal is the target user, the mobile phone performs speech recognition on the cached audio signal based on the target moment, and finally executes the control instruction corresponding to the speech recognition result. It can be seen that the control method of the present application performs wake-up word detection, voiceprint recognition and speech recognition on the same audio signal in sequence. When the user needs to control the device by voice, there is no need to perform voice wake-up and speech recognition respectively through two audio signals. As a result, the number of human-computer interactions is reduced, thereby improving the efficiency of device control.

[0169] Accordingly, the present application also provides a device control device (hereinafter referred to as the control device), see Fig.13 , Fig.1330 is a schematic diagram of the structure of the device control device provided in the present application. The control device 30 can be integrated in an intelligent terminal, and specifically can include a collection module 301, a detection module 302, a determination module 303, an identification module 304 and an execution module 305, as follows:

[0170] The acquisition module 301 is used to acquire the audio signal of the current environment;

[0171] For example, in flight mode, the acquisition module 301 can convert the collected sound into an analog electrical signal, and then the analog electrical signal needs to be digitized and converted into a digital signal that can be recognized by the computer. Optionally, in some embodiments, a wake-up chip is installed in the acquisition module 301, and the wake-up chip carried therein will continuously monitor the sounds of the surrounding environment and cache audio of a certain length. In this embodiment, the audio signal of the current environment is monitored by the wake-up chip to avoid frequent calls to the central processing unit to increase power consumption. Therefore, in this embodiment, the use of the wake-up chip to collect the audio signal of the current environment can reduce the power consumption of the device control device.

[0172] The detection module 302 is used to detect whether the audio signal contains a preset signal.

[0173] Optionally, the preset signal may be a wake-up word, a piece of music or an impact sound, etc. The wake-up word may be a word, a single word, a letter, a number or a character, etc. pre-set by the user.

[0174] Optionally, in some embodiments, the detection module 302 may specifically include:

[0175] A first determining unit, configured to determine or identify a wake-up mode corresponding to a current moment;

[0176] The detection unit is used to detect whether the audio signal contains a wake-up word according to the wake-up mode.

[0177] Optionally, in some embodiments, the detection module 302 can be specifically used to: if the wake-up mode is the first wake-up mode, select a target signal from a preset signal set, and detect whether the audio signal contains a signal identical to the target signal; and / or, if the wake-up mode is the second wake-up mode, detect whether the audio signal contains a signal similar to a signal in the preset signal set.

[0178] Optionally, in some embodiments, the audio duration includes a start detection moment, a first detection moment, a second detection moment, and an end detection moment, and optionally, the first detection moment is later than the start detection moment, the second detection moment is later than the first detection moment, and the end detection moment is later than the second detection moment. The detection module 302 can be specifically used to: determine the audio duration corresponding to the audio signal, and detect whether the audio signal contains a preset signal according to the wake-up mode between the start detection moment and the first detection moment; and / or, between the second detection moment and the end detection moment, detect whether the audio signal contains a preset signal according to the wake-up mode.

[0179] Optionally, in some embodiments, see Fig.14 The control device 30 may further include an adjustment module 306, wherein the adjustment module 306 may be specifically used to: detect a time interval corresponding to the current moment, and adjust the wake-up value corresponding to the current moment to a wake-up threshold corresponding to the time interval.

[0180] The determination module 303 is used to determine the time containing the preset signal as the target time.

[0181] The recognition module 304 is used to perform speech recognition on the audio signal buffered at the previous moment based on the target moment.

[0182] For example, optionally, the audio text corresponding to the audio signal is "How's the weather today, Xiaoyin", where "Xiaoyin" is the wake-up word, and the moment corresponding to "Xiaoyin" is determined as the target moment. In some embodiments, if multiple identical wake-up words appear in the audio signal, such as "Xiaoyin, how's the weather, Xiaoyin", the moment of the first wake-up word can be determined as the target moment, or the moment of the last wake-up word can be determined as the target moment; of course, the moment of the first wake-up word can also be determined as the first target moment, and the moment of the last wake-up word can be determined as the second target moment, such as "Xiaoyin Xiaoyin, help me turn up the screen brightness, Xiaoyin, check the weather forecast again", and voice recognition can be performed based on the first target moment and the second target moment in the future.

[0183] Optionally, in some embodiments, the identification module 304 may further include:

[0184] A first recognition unit, used for performing voiceprint recognition on an audio signal;

[0185] The second recognition unit is used for indicating that the source corresponding to the audio signal is a target source according to the voiceprint recognition result, and performing speech recognition on the audio signal cached at the previous moment.

[0186] Optionally, in some embodiments, the second identification unit may specifically include:

[0187] A second determining unit, configured to determine or output a source type corresponding to the target source, the source corresponding to the audio signal indicated by the voiceprint recognition result being the target source;

[0188] A third determining unit, configured to determine or identify a speech recognition interval including a target time;

[0189] The recognition unit is used to perform speech recognition on the buffered audio signal within the speech recognition interval based on the determined or output source type.

[0190] Optionally, the speech recognition interval includes a speech recognition start point and a speech recognition end point, and the target time is between the speech recognition start point and the speech recognition end point.

[0191] Optionally, in some embodiments, the recognition unit can be specifically used to: when the source type is a restricted source type, obtain a preset voice library, convert the cached audio signal into an audio text, calculate or determine the similarity between the audio text and at least one reference text; and determine the semantics corresponding to the reference text with the greatest similarity as the voice recognition result of the cached audio signal.

[0192] Optionally, the preset voice library includes at least one reference text. The reference text in the preset voice library can be pre-entered by the manufacturer or by the user. Of course, a basic voice library can be pre-entered before the device leaves the factory, and the user can record new voices on the basic voice library to form a preset voice library.

[0193] The execution module 305 is used to execute the control instruction corresponding to the speech recognition result.

[0194] Optionally, in some embodiments, the execution module 305 may be specifically used to: start the target application, display the application interface, and / or call the device component of the target device.

[0195] Optionally, in some embodiments, the execution module 305 can be specifically used to: obtain a user account currently logged in to the device, collect a voice control record of the user account, and execute a control instruction corresponding to the voice recognition result according to the voice control record.

[0196] Optionally, in the device control apparatus provided by the present application, the acquisition module 301 acquires the audio signal of the current environment, and then the detection module 302 detects whether the audio signal contains a preset signal, the determination module 303 determines the moment containing the preset signal as the target moment, and then the recognition module 304 uses the target moment as a reference to perform voice recognition on the audio signal cached at the previous moment, and finally, the execution module 305 executes the control instruction corresponding to the voice recognition result. It can be seen that the control method of the present application, when it is detected that the audio signal contains a preset signal, determines the moment containing the preset signal as the target moment, and uses the target moment as a reference to perform voice recognition on the audio signal cached at the previous moment. Through the above technical solution, when the user needs to control the device by voice, there is no need to respectively perform the functions of voice wake-up and voice recognition through two audio signals, thereby reducing the number of human-computer interactions, thereby improving the efficiency of device control.

[0197] The present application also provides an intelligent terminal, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the steps of the control method in any of the above embodiments are implemented.

[0198] The present application also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the control method in any of the above embodiments are implemented.

[0199] In the embodiments of the smart terminal and the readable storage medium provided in the present application, all the technical features of any of the above-mentioned control method embodiments may be included, and the expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods, and will not be repeated here.

[0200] The embodiment of the present application further provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes the methods in the above various possible implementation modes.

[0201] An embodiment of the present application also provides a chip, including a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device equipped with the chip executes the methods in various possible implementation modes as described above.

[0202] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0203] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0204] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0205] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0206] In the present application, the same or similar terminology concepts, technical solutions and / or application scenario descriptions are generally described in detail only the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of the present application, for the same or similar terminology concepts, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0207] In the present application, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0208] The various technical features of the technical solution of the present application can be arbitrarily combined. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0209] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

[0210] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium, (e.g., a floppy disk, a storage disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk Solid State Disk (SSD)), etc.

[0211] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A control method, characterized in that: include: S10: collecting an audio signal of the current environment, and detecting whether the audio signal contains a preset signal; S20: Determine the moment containing the preset signal as the target moment, and perform speech recognition on the audio signal buffered at the previous moment based on the target moment; Before step S20, it includes: performing voiceprint recognition on the audio signal, the voiceprint recognition result indicates that the source corresponding to the audio signal is the target source, and performing speech recognition on the audio signal cached at the previous moment; the voiceprint recognition result indicates that the source corresponding to the audio signal is the target source, and performing speech recognition on the audio signal cached at the previous moment, including: the voiceprint recognition result indicates that the source corresponding to the audio signal is the target source, determining or outputting the source type corresponding to the target source, determining or identifying the speech recognition interval containing the target moment, and based on the determined or output source type, performing speech recognition on the cached audio signal within the speech recognition interval; based on the determined or output source type, performing speech recognition on the cached audio signal within the speech recognition interval, including: the source type is a restricted source type, obtaining a preset voice library, the preset voice library including at least one reference text, converting the cached audio signal into an audio text, calculating or determining the similarity between the audio text and at least one reference text, and determining the semantics corresponding to the reference text with the greatest similarity as the speech recognition result of the cached audio signal; S30: Execute a control instruction corresponding to the speech recognition result.

2. The method according to claim 1, characterized in that The step S10 includes: Determine or identify the wake-up mode corresponding to the current moment; and detecting whether the audio signal includes a preset signal according to the wake-up mode.

3. The method according to claim 2, characterized in that The detecting, according to the wake-up mode, whether the audio signal includes a preset signal comprises: The wake-up mode is a first wake-up mode, a target signal is selected from a preset signal set, and it is detected whether the audio signal contains a signal identical to the target signal; and / or, The wake-up mode is a second wake-up mode, and detects whether the audio signal includes a signal similar to a signal in the preset signal set.

4. The method according to claim 2, characterized in that The detecting, according to the wake-up mode, whether the audio signal includes a preset signal comprises: Determine or identify an audio duration corresponding to the audio signal, where the audio duration includes a start detection time, a first detection time, a second detection time, and an end detection time; From the start detection time to the first detection time, detecting whether the audio signal contains a preset signal according to the wake-up mode; and / or, From the second detection time to the termination detection time, detecting whether the audio signal includes a preset signal according to the wake-up mode.

5. The method according to any one of claims 1 to 4, characterized in that Also includes: Detect the time interval corresponding to the current moment; The wake-up value corresponding to the current moment is adjusted to the wake-up threshold corresponding to the time interval.

6. An intelligent terminal, characterized in that: The intelligent terminal comprises: a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the control method according to any one of claims 1 to 5 are implemented.

7. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the control method according to any one of claims 1 to 5 are implemented.

Images