Control method and device of electronic equipment and electronic equipment
By evaluating audio data in stages and controlling processor status, the problem of voice wake-up time of smart devices is solved, reducing wake-up time and power consumption, and improving user experience.
Patent Information
- Application Number
- CN202411216762.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-03
AI Technical Summary
It takes a long time to switch from sleep to running when the existing smart devices wake up, affecting the user experience.
By evaluating and detecting the audio data collected by the wake-up engine in two stages, the application processor is controlled to enter standby, ready and active states according to different detection results, avoiding false wake-up and reducing wake-up time and power consumption.
It effectively reduces the wake-up time of electronic devices, reduces the power consumption generated during monitoring audio data and audio recognition, and improves the user experience.
Smart Images

Figure CN119065484A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of terminal technology, and in particular to a control method and device for an electronic device and the electronic device. Background Art
[0002] Voice wake-up is to activate the device from sleep state to running state through voice. At present, when voice wake-up is performed, it takes a long time for smart devices to switch from sleep state to running state, which affects the user experience. Summary of the invention
[0003] In view of this, the present disclosure provides a control method and device for an electronic device, and the electronic device.
[0004] According to a first aspect of the present disclosure, a control method for an electronic device is provided, comprising: in response to a first evaluation value of the electronic device for audio data collected by a wake-up engine at a first moment being greater than or equal to a first threshold, controlling an application processor of the electronic device to enter a ready state from a standby state; and in response to a second evaluation value of the electronic device for audio data collected by the wake-up engine at a second moment being greater than or equal to a second threshold, controlling the application processor to enter an active state from a ready state; wherein the evaluation value represents a degree of match between the audio data collected by the wake-up engine of the electronic device and a wake-up word of the electronic device, the second threshold is greater than the first threshold, the power consumption of the electronic device in the standby state is less than that in the ready state, and the power consumption in the ready state is less than that in the active state.
[0005] According to an embodiment of the present disclosure, in response to a second evaluation value of the electronic device for the audio data collected by the wake-up engine at the second moment being greater than or equal to a second threshold, controlling the application processor to enter an active state from a ready state includes: in response to the audio collected by the electronic device for the wake-up engine at the second moment being a complete wake-up word, controlling the application processor to enter an active state from a ready state; or in response to the second evaluation value of the electronic device for the audio data collected by the wake-up engine at the second moment being greater than or equal to a second threshold, and the evaluation value increases between the first moment and the second moment, controlling the application processor to enter an active state from a ready state.
[0006] According to an embodiment of the present disclosure, the method further includes: in response to the evaluation value of the electronic device for the audio data collected by the wake-up engine at the second moment being less than a second threshold, controlling the application processor to return from the ready state to the standby state.
[0007] According to an embodiment of the present disclosure, the method further includes: in response to the application processor entering an active state from a ready state, controlling the wake-up engine to enter a second working mode from a first working mode, wherein power consumption of the second working mode is lower than that of the first working mode.
[0008] According to an embodiment of the present disclosure, in response to a second evaluation value of the electronic device for audio data collected by the wake-up engine at a second moment being greater than or equal to a second threshold, controlling the application processor to enter an active state from a ready state includes: in response to a third evaluation value of the electronic device for audio data collected by the wake-up engine at a third moment being greater than or equal to the second threshold, and the evaluation values from the third moment to the second moment are all greater than the second threshold, controlling the application processor to enter an active state from a ready state, wherein the third moment is earlier than the second moment.
[0009] According to an embodiment of the present disclosure, the duration from the third moment to the second moment varies with the duration of the wake-up word.
[0010] According to an embodiment of the present disclosure, the method also includes: acquiring in real time ambient audio collected by the wake-up engine, wherein the ambient audio is an expression of the real environment; extracting audio data from the ambient audio, wherein the audio data is voice information of the target user; and calculating an evaluation value of the audio data at the current moment, wherein the evaluation value represents the degree of match between the audio data and the sound features of the wake-up word.
[0011] According to an embodiment of the present disclosure, the method further includes: in response to the application processor entering an active state, controlling the electronic device to display a user interface and waking up a target application in the electronic device, wherein the target application is used to enable voice interaction between the target user and the electronic device.
[0012] A second aspect of the present disclosure provides a control device for an electronic device, comprising: a first control module, for controlling an application processor of the electronic device to enter a ready state from a standby state in response to a first evaluation value of audio data collected by a wake-up engine of the electronic device at a first moment being greater than or equal to a first threshold; and a second control module, for controlling the application processor to enter an active state from a ready state in response to a second evaluation value of audio data collected by the wake-up engine of the electronic device at a second moment being greater than or equal to a second threshold; wherein the evaluation value represents a degree of match between the audio data collected by the wake-up engine of the electronic device and a wake-up word of the electronic device, the second threshold is greater than the first threshold, the power consumption of the electronic device in the standby state is less than that in the ready state, and the power consumption in the ready state is less than that in the active state.
[0013] The third aspect of the present disclosure provides an electronic device, comprising: a wake-up engine; a memory for storing one or more programs, and one or more processors, so that when the one or more programs are executed, the wake-up engine and the one or more processors execute any one of the methods described in the first aspect.
[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0016] Figure 1 A flowchart schematically shows a method for controlling an electronic device according to an embodiment of the present disclosure;
[0017] Figure 2 A schematic diagram schematically shows a control method of an electronic device according to an embodiment of the present disclosure;
[0018] Figure 3 A flowchart of a control method for an electronic device according to an embodiment of the present disclosure is schematically shown;
[0019] Figure 4 A flowchart schematically shows a method for controlling an electronic device according to another embodiment of the present disclosure;
[0020] Figure 5 A flowchart of a method for determining audio data according to another embodiment of the present disclosure is schematically shown;
[0021] Figure 6 A flowchart schematically shows a method for controlling an electronic device according to another embodiment of the present disclosure;
[0022] Figure 7 A schematic diagram of a control method of an electronic device according to an embodiment of the present disclosure is shown;
[0023] Figure 8 A structural block diagram schematically shows a control device of an electronic device according to an embodiment of the present disclosure; and
[0024] Fig. 9 A block diagram of an electronic device suitable for implementing a control method of an electronic device according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0026] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0027] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0028] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0029] The embodiments of the present disclosure provide a control method and device for an electronic device and the electronic device. Before introducing the technical solutions provided by the embodiments of the present disclosure, the related technologies involved in the present disclosure are first described.
[0030] Among the related technologies, voice interaction technology is a relatively important technology in current electronic devices, including smart phones, speakers, televisions, robots, tablet devices, car-mounted devices, etc. The voice wake-up function is one of the key functions of voice interaction technology. Through specific wake-up words or command words (such as "Xiao Le Xiao Le"), electronic devices in non-voice interaction states (such as sleep state or other states) are activated. The awakened electronic devices will respond to the speaker's request, interact with the user, and enable other voice functions such as voice recognition, voice search, conversation, and voice navigation.
[0031] Currently, when voice wake-up is performed, it takes a long time for smart devices to switch from sleep mode to running mode, which affects the user experience.
[0032] Specifically, voice wake-up requires continuous collection of voice data and matching inference operations with the preset wake-up words of the electronic device. On electronic devices, in order to reduce power consumption, the above-mentioned voice data collection and operation are generally run on a separate digital signal processing (DSP) chip. When the user does not interact with the device for a long time, the processor (Application Processor, AP) of the electronic device enters a sleep state (Suspend). After the DSP detects voice wake-up, it wakes up the processor, starts the service, lights up the screen and starts the corresponding wake-up application (for example: voice assistant). It takes a long time for the processor to switch from sleep state (standby state) to running state, which affects the timely response to the user's wake-up operation.
[0033] Before further describing the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations.
[0034] Multi-source separation technology: refers to the technology of separating the received single-microphone or multi-microphone voice signal into multiple sound source data. Among them, the multiple sound source data include the sound source data of the target object and the sound source data of the interfering sound source. Multi-source separation technology is used to separate the sound source data of the target object from the sound source data of the interfering sound source in order to better perform wake-up judgment.
[0035] Voice wake-up technology can be divided into keyword detection technology and voiceprint recognition technology according to the different recognition objects. Among them, keyword detection technology refers to the technology of identifying specific voice commands (i.e. keywords, also known as wake-up words) in a continuous voice stream. Voiceprint recognition (also known as user recognition) technology refers to the technology of extracting user voice features and verifying user identity. This technology is often used in identity authentication, which can ensure the security of personal information and improve user experience.
[0036] An embodiment of the present disclosure provides a control method for an electronic device, comprising: in response to a first evaluation value of the electronic device for audio data collected by a wake-up engine at a first moment being greater than or equal to a first threshold, controlling an application processor of the electronic device to enter a ready state from a standby state; and in response to a second evaluation value of the electronic device for audio data collected by the wake-up engine at a second moment being greater than or equal to a second threshold, controlling the application processor to enter an active state from a ready state; wherein the evaluation value represents the degree of matching between the audio data collected by the wake-up engine of the electronic device and the wake-up word of the electronic device, the second threshold is greater than the first threshold, the power consumption of the electronic device in the standby state is less than that in the ready state, and the power consumption in the ready state is less than that in the active state. On the one hand, since the method evaluates and detects the audio data collected by the wake-up engine in two stages, the application processor is controlled to enter different states according to different detection results, which can prevent the electronic device from being woken up by mistake. On the other hand, the two staged detections correspond to two different working states (ready state and active state) of the application processor, respectively. When the detection result meets the target result, the application processor is controlled to change different working states, so that the power consumption of the application processor is increased in sequence, which can not only reduce the wake-up time of the electronic device, but also reduce the power consumption generated by the device monitoring audio data and performing audio recognition.
[0037] The following will be passed Figure 1 to Figure 7 A method for controlling an electronic device according to an embodiment of the disclosure is described in detail.
[0038] The electronic device may be a smart speaker, a smart robot or other device, and may also be a smart phone, a tablet computer, a desktop / laptop / handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) / virtual reality (VR) device or other device including a touch screen. The embodiments of the present disclosure are not intended to limit the specific form of the electronic device.
[0039] Figure 1 The flowchart of the control method of the electronic device according to the embodiment of the present disclosure is schematically shown. Figure 2 A schematic diagram schematically shows a control method of an electronic device according to an embodiment of the present disclosure.
[0040] like Figure 1 As shown, the control method of the electronic device of this embodiment includes operation S110 to operation S120.
[0041] In operation S110, in response to a first evaluation value of the electronic device for audio data collected by the wake-up engine at a first moment being greater than or equal to a first threshold, an application processor of the electronic device is controlled to enter a ready state from a standby state.
[0042] The evaluation value represents the degree of match between the audio data collected by the wake-up engine of the electronic device and the wake-up word of the electronic device.
[0043] Exemplarily, the audio data collected by the wake-up engine can be obtained by real-time collection of the current scene of the device through hardware such as a microphone of the electronic device. Of course, it can also be transmitted to the device in real time through mobile network communication and other methods.
[0044] The wake-up engine performs wake-up detection based on the audio data collected at the first moment, and when the degree of match between the audio data and the wake-up word pre-stored in the electronic device is greater than or equal to the first threshold, the application processor of the electronic device is pre-awakened (ready state). Figure 2 In (a) and (b), the wake-up word is "Xiao Le Xiao Le", and the matching degree between the audio data and the wake-up word is set to 0-1.2. The application processor of the electronic device is pre-awakened only when the matching degree between the words in the audio data (e.g., Xiao Le, Xiao Le Xiao) and the wake-up word reaches 0.6, and the audio data that does not contain "Xiao Le" is eliminated and blocked.
[0045] In the standby state, the electronic device has no ability to interact with the user by voice. The power consumption of the electronic device in the standby state is less than that in the standby state.
[0046] It should be noted that the matching degree between the audio data and the wake-up word here can be the matching degree of the wake-up word content, or the matching degree of the wake-up word content plus voiceprint recognition.
[0047] In operation S120, in response to the electronic device having a second evaluation value of the audio data collected by the wake-up engine at the second moment being greater than or equal to a second threshold, the application processor is controlled to enter an active state from a ready state.
[0048] The second threshold is greater than the first threshold, and the power consumption in the ready state is less than the power consumption in the active state.
[0049] Exemplarily, due to the real-time nature of audio collection, the contents of the audio data collected by the engine awakened at the first moment and the second moment may be different.
[0050] The wake-up engine performs a wake-up detection based on the audio data collected at the second moment, and when the degree of match between the audio data and the wake-up word pre-stored in the electronic device is greater than or equal to the second threshold, the application processor of the electronic device is truly awakened (active state). For example, when the degree of match between the words in the audio data and the wake-up word reaches 1, the application processor of the electronic device is truly awakened.
[0051] In the active state, the electronic device has the ability to interact with the user by voice. The power consumption of the electronic device in the ready state is less than that in the active state.
[0052] It should be noted that the embodiments of the present disclosure do not impose any specific limitation on the settings of the first threshold and the second threshold, and the numerical values of the first threshold and the second threshold can be adjusted according to design requirements.
[0053] Monitor the audio data and perform target wake-up word detection on the monitored audio data. First, perform a first evaluation on the degree of match between the audio data and the target wake-up word, and when it is determined that the first evaluation result is the first target result, control the electronic device application processor to enter the ready state from the standby state. Then perform a second evaluation on the degree of match between the audio data and the target wake-up word, and when it is determined that the second evaluation result is the second target result, control the electronic device application processor to enter the active state from the ready state, and the power consumption of the electronic device increases from the standby state to the ready state and then to the active state.
[0054] It can be understood that, on the one hand, since the method evaluates and detects the audio data collected by the wake-up engine in two stages, the application processor is controlled to enter different states according to different detection results, which can prevent the electronic device from being woken up by mistake. On the other hand, the two-stage detection corresponds to two different working states of the application processor (ready state and active state). When the detection result meets the target result, the application processor is controlled to change to a different working state, so that the power consumption of the application processor increases successively, which can not only reduce the wake-up time of the electronic device, but also reduce the power consumption generated by the device monitoring audio data and performing audio recognition.
[0055] As described above, in operation S120, in response to the second evaluation value of the electronic device for the audio data collected by the wake-up engine at the second moment being greater than or equal to the second threshold, the application processor is controlled to enter the active state from the ready state. In an achievable manner, the operation may further include: in response to the audio collected by the electronic device for the wake-up engine at the second moment being a complete wake-up word, the application processor is controlled to enter the active state from the ready state.
[0056] In the embodiment of the present disclosure, when the application processor is in the ready state, if the words in the audio data collected by the wake-up engine at the second moment contain the complete wake-up word, it means that the second evaluation value of the audio data at the second moment is greater than or equal to the second threshold, so that the application processor of the electronic device enters the active state from the ready state. For example, the complete wake-up word is "Xiao Le Xiao Le", and "Xiao Le Xiao Le" is detected in the words of the audio data, which is consistent with the complete wake-up word, and the application processor is truly awakened.
[0057] As described above, in operation S120, in response to the second evaluation value of the electronic device for the audio data collected by the wake-up engine at the second moment being greater than or equal to the second threshold, the application processor is controlled to enter the active state from the ready state. In another achievable manner, the operation may further include: in response to the second evaluation value of the electronic device for the audio data collected by the wake-up engine at the second moment being greater than or equal to the second threshold, and the evaluation value increases between the first moment and the second moment, the application processor is controlled to enter the active state from the ready state.
[0058] In the embodiment of the present disclosure, when the application processor is in the ready state, the matching degree (second evaluation value) between the words in the audio data collected by the wake-up engine and the wake-up word at the second moment is greater than or equal to the second threshold, and the overall evaluation value of the audio data between the first moment and the second moment is in an increasing state, so that the application processor of the electronic device enters the active state from the ready state. For example, referring to Figure 2 In (c), the application processor of the electronic device is actually woken up only when the degree of match between the words in the audio data and the wake-up words reaches 1 at the second moment, and the degree of match (evaluation value) between the words in the audio data collected between the first moment and the second moment and the wake-up words is on an upward trend as a whole.
[0059] It should be noted that the wake-up word in the audio data can be complete or incomplete. For example, if the word in the audio data is "xiaole xiaolang", since the first starting sound of "lang" and "le" are the same, the evaluation value can be equal to 1 when calculating the evaluation value.
[0060] It is understandable that judging the audio data based on the first preset condition (whether it contains a complete wake-up word) or the second preset condition (whether the evaluation value meets the preset threshold and increases) can make the wake-up judgment result more accurate.
[0061] Figure 3 The following schematically shows a judgment flow chart of a control method of an electronic device according to an embodiment of the present disclosure.
[0062] The control method of the electronic device may further include operations S210 to S250 ′.
[0063] Among them, operations S210 to S230 refer to the above-mentioned operation S110, and operations S240 to S250 refer to the above-mentioned operation S120, which will not be repeated here.
[0064] In operation S250 ′, in response to the second evaluation value of the electronic device for the audio data collected by the wake-up engine at the second moment being less than the second threshold, the application processor is controlled to return from the ready state to the standby state.
[0065] In the embodiments of the present disclosure, Figure 3 As shown, when the application processor is in the ready state, if the wake-up engine performs a wake-up detection based on the audio data collected at the second moment, and the degree of match between the audio data and the wake-up word pre-stored in the electronic device is less than the second threshold, the application processor of the electronic device is returned from the ready state to the standby state. For example, at the first moment, when the degree of match between the words in the audio data and the wake-up word is detected to reach 0.6, the application processor enters the ready state from the standby state. At the second moment, when the degree of match between the words in the audio data and the wake-up word is detected to be less than 1, it means that the words in the audio data do not meet the conditions for the application processor to actually wake up, and the user does not want to wake up the electronic device, so the application processor of the electronic device is switched to the standby state with the lowest power consumption.
[0066] It is understandable that when the application processor of the electronic device in the standby state does not meet the conditions for actual awakening, returning the application processor of the electronic device to the standby state with the lowest power consumption can reduce the power consumption of the electronic device.
[0067] In some other embodiments, at operation 230 ′, in response to a first evaluation value of the electronic device for the audio data collected by the wake-up engine at the first moment being less than a first threshold, the application processor is controlled to remain in the standby state.
[0068] For example, continue to refer to Figure 2 If the detection result of the audio data at the first moment is: the matching degree between the words in the audio data and the wake-up words is less than 0.6, the condition for the application processor of the electronic device to enter the ready state is not met, and there is no need to pre-wake up the application processor of the electronic device, so that the application processor of the electronic device can be kept in the standby state with the lowest power consumption.
[0069] As described above, the control method of the electronic device may further include: operation 260, in response to the application processor entering the active state from the ready state, controlling the wake-up engine to enter the second working mode from the first working mode, and the power consumption of the second working mode is lower than that of the first working mode.
[0070] In the embodiments of the present disclosure, continue to refer to Figure 2 and Figure 3 In the process of the application processor entering the active state from the ready state, the working mode of the wake-up engine enters the second working mode from the first working mode, and the power consumption of the second working mode is lower than that of the first working mode.
[0071] The first working mode can be the full working state of the wake-up engine, that is, the wake-up engine collects audio data and calculates the evaluation value. The second working mode can be the non-working state of the wake-up engine, that is, the wake-up engine no longer collects audio data and calculates the evaluation value (the power consumption of the wake-up engine becomes smaller), but the application processor collects and processes the audio data, and the wake-up engine no longer wakes up the application processor. For example, after the application processor enters the active state from the ready state, the application processor and the user are in a state of voice interaction. At this time, if the user says the voice of "Xiao Le Xiao Le", the wake-up engine no longer replies "I am here", but the application processor replies.
[0072] In other embodiments, in response to the application processor entering the active state from the ready state, the wake-up engine is controlled to maintain the first working mode unchanged.
[0073] For example, after the application processor enters the active state from the ready state, the application processor and the user are in a state of voice interaction. At this time, if the user says "Xiao Le Xiao Le", the voice interaction between the application processor and the user will be interrupted, and the wake-up engine will reply "I am here", and the application processor will be woken up again, so that the user and the electronic device will enter the state of voice interaction again.
[0074] As described above, in operation S120, in response to the electronic device having a second evaluation value of the audio data collected by the wake-up engine at the second moment being greater than or equal to the second threshold, the application processor is controlled to enter the active state from the ready state. In one achievable manner, the operation may further include: in response to the electronic device having a third evaluation value of the audio data collected by the wake-up engine at the third moment being greater than or equal to the second threshold, and the evaluation values from the third moment to the second moment are all greater than the second threshold, the application processor is controlled to enter the active state from the ready state, wherein the third moment is earlier than the second moment.
[0075] In the embodiment of the present disclosure, in order to ensure the accuracy of voice wake-up of the electronic device, when calculating the evaluation value of the audio data, if it is detected at a third moment before the second moment that the evaluation value of the audio data at the third moment is greater than or equal to the second threshold, and the evaluation value of the audio data between the third moment and the second moment is always greater than or equal to the second threshold, the application processor is made to enter the active state (real wake-up) from the ready state.
[0076] For example, taking the words "Xiaole Xiaolang" in the above audio data as an example, the evaluation value of the audio data detected at the third moment is equal to 1. To avoid waking up the application processor in this case, it is necessary to ensure that the evaluation value of the audio data is always greater than or equal to 1 for a period of time after the third moment (i.e., from the third moment to the second moment) before the application processor can be truly woken up, so as to avoid false wake-up. Continuing to refer to Figure 2 In (c) of, if the words in the audio data are "Xiaole Xiaole", then when calculating the fourth character "Le", the evaluation value is always greater than the second threshold, and the application processor can be truly woken up.
[0077] In some embodiments, the duration from the third moment to the second moment changes with the duration of the wake-up word.
[0078] It should be known that in the process of audio signal processing, the audio signal is usually divided into smaller segments or windows. Different users speak at different frequencies when saying the wake-up word "Xiaole Xiaole", that is, the duration of saying the entire wake-up word is different. When calculating the evaluation value based on multiple segments of the audio signal, the definition of the duration from the third moment to the second moment needs to be based on the duration of the user saying the wake-up word (speaking frequency). When a person speaks slowly (low speaking frequency), the duration for which the detection evaluation value is always greater than the second threshold needs to be longer; when a person speaks fast (high speaking frequency), the duration for which the detection evaluation value is always greater than the second threshold needs to be shorter.
[0079] It can also be understood that if the content of the wake-up word increases and the length of the wake-up word changes, such as "hi Xiaole Xiaole", the duration for the user to say the wake-up word may increase. When detecting the matching degree between the words in the audio data and the wake-up word, correspondingly, the detection duration will also increase. Therefore, the duration from the third moment to the second moment will also be extended.
[0080] It should be noted that the present disclosure embodiments do not make specific limitations on the magnitude of the duration from the third moment to the second moment. The duration that meets the user's needs can be set according to the requirements of the actual product.
[0081] Figure 4 Schematically shows a flowchart of a control method for an electronic device according to another embodiment of the present disclosure.
[0082] As described above, for the acquisition of audio data in operations S110 to S120, in one implementable manner, as Figure 4 shown, it may include operations S310 to S330.
[0083] In operation S310, the ambient audio collected by the wake-up engine is acquired in real time, where the ambient audio is an expression of the real environment.
[0084] In operation S320, audio data is extracted from the ambient audio, wherein the audio data is voice information of the target user.
[0085] In operation S330, an evaluation value of the audio data at the current moment is calculated, wherein the evaluation value represents a degree of matching between the audio data and the sound features of the wake-up word.
[0086] Exemplarily, the ambient audio is the sound generated in the real environment that the electronic device is currently in. The ambient audio includes audio data of the target user and sound source data of the interfering sound source, and the interfering sound source includes at least one of the speech of other objects other than the target user, background music, and ambient noise.
[0087] When the wake-up engine calculates the audio data, it performs the corresponding calculation based on the audio data of the target user. Therefore, after the wake-up engine collects the ambient audio through the microphone, it can separate the audio data corresponding to the target user from the audio data generated by other target objects based on the characteristics of the sounds of different target objects (for example, voiceprint, volume). Only the audio data of the target user is used to calculate the evaluation value.
[0088] It is understandable that extracting the audio data of the target user from the collected ambient audio for independent calculation and analysis can avoid the influence of other sounds on the calculation of the audio data evaluation value, thereby improving the accuracy of the evaluation value calculation.
[0089] Figure 5 The flowchart of a method for determining audio data according to another embodiment of the present disclosure is schematically shown.
[0090] As described above, in operation S320, audio data is extracted from the ambient audio, wherein the audio data is the voice information of the target user. Figure 5 As shown, the operation may further include operation S321 to operation S323.
[0091] In operation S321, sound source separation processing is performed on the ambient audio to obtain voice information of at least one user.
[0092] In operation S322, first voiceprint information of at least one user is determined based on the voice information.
[0093] In operation S323, when the first voiceprint information matches the second voiceprint information preset in the electronic device, the user corresponding to the first voiceprint information is determined as the target user, and the voice information of the user is audio data.
[0094] Among them, the sound source represents the sound produced by different target objects.
[0095] In the disclosed embodiment, the ambient audio obtained by the electronic device based on the real environment in which it is located is separated according to different target objects to obtain voice information corresponding to multiple target objects (including audio data of the target user). Based on the voice information, the first voiceprint information of the multiple target objects is determined. Since the wake-up audio (including the second voiceprint information) may be pre-stored in the electronic device, the user can wake up the application processor only when the first voiceprint information matches the second voiceprint information. Whether the target object is the target user is determined based on whether the multiple first voiceprint information matches the second voiceprint information. Only when the target object is the target user, the audio data of the target object (target user) will be further analyzed for wake-up.
[0096] Figure 6 The flowchart of a control method of an electronic device according to another embodiment of the present disclosure is schematically shown.
[0097] As described above, the control method of the electronic device may further include: in response to the application processor entering an active state, controlling the electronic device to display a user interface and waking up a target application in the electronic device, wherein the target application is used to enable voice interaction between the target user and the electronic device.
[0098] In the embodiments of the present disclosure, refer to Figure 6 When the application processor is truly awakened (enters the active state from the ready state), the application processor enters the normal working mode, and the screen of the electronic device is lit. Then the wake-up event is sent to the upper-layer application to start the voice assistant (target application) of the electronic device, so that the user interacts with the voice assistant through voice. If the application processor is not truly awakened (does not enter the active state from the ready state), the voice wake-up service is stopped and the application processor is put into the standby state.
[0099] In order to facilitate the understanding of the control method of the electronic device in the embodiment of the present disclosure, Figure 7 Provide further explanation.
[0100] Figure 7 The principle diagram of the control method of the electronic device according to the embodiment of the present disclosure is schematically shown.
[0101] like Figure 7 As shown, the wake-up engine (digital signal processor, DSP) of the electronic device collects the ambient audio (original audio) in real time through the microphone and inputs it into the audio processing engine. The audio processing engine performs echo cancellation and noise suppression on the ambient audio to obtain audio data. The wake-up engine performs wake-up word detection on the audio data to obtain the matching degree (evaluation value) between the words in the audio data at different times and the wake-up word. That is, pre-wake-up detection and wake-up detection are performed.
[0102] The wake-up engine performs a wake-up detection based on the audio data collected at the first moment. When the degree of match between the audio data and the wake-up word pre-stored in the electronic device is greater than or equal to the first threshold, the application processor of the electronic device is pre-awakened (ready state / shallow sleep mode), the device is in a low-power state, the LCD is turned off, and only some necessary modules such as the wake-up service are running, and the voice assistant is not awakened. Next, the wake-up engine performs a wake-up detection based on the audio data collected at the second moment. When the degree of match between the audio data and the wake-up word pre-stored in the electronic device is greater than or equal to the second threshold, the application processor of the electronic device is actually awakened (active state / normal working mode) to wake up the voice assistant.
[0103] Based on the above electronic device control method, the present disclosure also provides an electronic device control device. Figure 8 The device is described in detail.
[0104] Figure 8 The structure block diagram of the control device of the electronic device according to the embodiment of the present disclosure is schematically shown.
[0105] like Figure 8 As shown, the control device 400 of the electronic device of this embodiment includes a first control module 410 and a second control module 420 .
[0106] The first control module 410 is used to control the application processor of the electronic device to enter the ready state from the standby state in response to the first evaluation value of the electronic device for the audio data collected by the wake-up engine at the first moment being greater than or equal to the first threshold. In one embodiment, the first control module 410 can be used to perform the operation S110 described above, which will not be repeated here.
[0107] The second control module 420 is used to control the application processor to enter the active state from the ready state in response to the second evaluation value of the electronic device for the audio data collected by the wake-up engine at the second moment being greater than or equal to the second threshold. In one embodiment, the second control module 420 can be used to perform the operation S120 described above, which will not be repeated here.
[0108] Among them, the evaluation value represents the degree of match between the audio data collected by the wake-up engine of the electronic device and the wake-up word of the electronic device, the second threshold is greater than the first threshold, the power consumption of the electronic device in the standby state is less than that in the ready state, and the power consumption in the ready state is less than that in the active state.
[0109] In some embodiments, the second control module includes: a first control submodule.
[0110] In one achievable manner, the first control submodule is used to control the application processor to enter an active state from a ready state in response to the audio collected by the electronic device for the wake-up engine at the second moment being a complete wake-up word.
[0111] In another possible implementation, the first control submodule is used to control the application processor to enter an active state from a ready state in response to a second evaluation value of the electronic device for the audio data collected by the wake-up engine at a second moment being greater than or equal to a second threshold, and the evaluation value increases between the first moment and the second moment.
[0112] In some embodiments, the second control module is further used to control the application processor to return from the ready state to the standby state in response to the evaluation value of the electronic device for the audio data collected by the wake-up engine at the second moment being less than a second threshold.
[0113] In some embodiments, the control device of the electronic device also includes a third control module, which is used to control the wake-up engine from the first working mode to the second working mode in response to the application processor entering the active state from the ready state, and the power consumption of the second working mode is lower than that of the first working mode.
[0114] In some embodiments, the second control module is used to control the application processor from a ready state to an active state in response to a third evaluation value of the electronic device for the audio data collected by the wake-up engine at a third moment being greater than or equal to a second threshold, and the evaluation values from the third moment to the second moment are all greater than the second threshold, wherein the third moment is earlier than the second moment.
[0115] In some embodiments, the duration from the third moment to the second moment varies with the duration of the wake-up word.
[0116] In some embodiments, the device further includes: an acquisition module, an extraction module and a calculation module.
[0117] The acquisition module is used to acquire the ambient audio collected by the wake-up engine in real time, where the ambient audio is an expression of the real environment.
[0118] The extraction module is used to extract audio data from the ambient audio, wherein the audio data is the voice information of the target user.
[0119] The calculation module is used to calculate the evaluation value of the audio data at the current moment, wherein the evaluation value represents the degree of match between the audio data and the sound features of the wake-up word.
[0120] In some embodiments, the device also includes a fourth control module, which is used to control the electronic device to display a user interface and wake up a target application in the electronic device in response to the application processor entering an active state, and the target application is used to enable voice interaction between the target user and the electronic device.
[0121] According to the exemplary embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the exemplary embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation.
[0122] According to an embodiment of the present disclosure, any multiple modules in the first control module 410 and the second control module 420 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first control module 410 and the second control module 420 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the first control module 410 and the second control module 420 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.
[0123] Fig. 9 A block diagram of an electronic device suitable for implementing a control method of an electronic device according to an embodiment of the present disclosure is schematically shown.
[0124] The embodiment of the present disclosure discloses an electronic device, including: a wake-up engine, a memory, and a processor.
[0125] The wake-up engine is used to collect audio data from users and determine the evaluation value based on the audio data.
[0126] A memory for storing one or more programs.
[0127] One or more processors are used to control the application processor of the electronic device to enter a ready state from a standby state in response to a first evaluation value of the audio data collected by the wake-up engine of the electronic device at a first moment being greater than or equal to a first threshold; and to control the application processor to enter an active state from a ready state in response to a second evaluation value of the audio data collected by the wake-up engine of the electronic device at a second moment being greater than or equal to a second threshold as a complete wake-up word; wherein the evaluation value represents the degree of match between the audio data collected by the wake-up engine of the electronic device at the first moment and the wake-up word associated with the state of the electronic device, the second threshold is greater than the first threshold, the power consumption of the electronic device in the standby state is less than that in the ready state, and the power consumption in the ready state is less than that in the active state.
[0128] According to the embodiment of the present disclosure, the processor can also be used to execute the control method of the electronic device described in other embodiments. For other embodiments, please refer to the relevant description of the control method of the electronic device provided in the above embodiment of the present disclosure, which will not be repeated here.
[0129] like Fig. 9 As shown, the electronic device 500 according to an embodiment of the present disclosure includes a wake-up engine (not shown in the figure) and a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 to a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
[0130] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.
[0131] According to an embodiment of the present disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.
[0132] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
[0133] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.
[0134] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the control method of the electronic device provided by the embodiment of the present disclosure.
[0135] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 501. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0136] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0137] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.
[0138] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).
[0139] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0140] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0141] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. A method for controlling an electronic device, comprising: In response to a first evaluation value of the electronic device for the audio data collected by the wake-up engine at a first moment being greater than or equal to a first threshold, controlling an application processor of the electronic device to enter a ready state from a standby state; as well as In response to a second evaluation value of the electronic device for the audio data collected by the wake-up engine at a second moment being greater than or equal to a second threshold, controlling the application processor to enter an active state from the ready state; Among them, the evaluation value represents the degree of match between the audio data collected by the wake-up engine of the electronic device and the wake-up word of the electronic device, the second threshold is greater than the first threshold, the power consumption of the electronic device in the standby state is less than that in the ready state, and the power consumption in the ready state is less than that in the active state.
2. The method according to claim 1, wherein: In response to a second evaluation value of the electronic device for the audio data collected by the wake-up engine at a second moment being greater than or equal to a second threshold, controlling the application processor to enter an active state from the ready state includes: In response to the electronic device detecting that the audio collected by the wake-up engine at the second moment is a complete wake-up word, controlling the application processor to enter an active state from the ready state; Or in response to a second evaluation value of the electronic device for the audio data collected by the wake-up engine at a second moment being greater than or equal to a second threshold, and the evaluation value increases between the first moment and the second moment, the application processor is controlled to enter an active state from the ready state.
3. The method according to claim 1, wherein: The method further comprises: In response to the electronic device's evaluation value of the audio data collected by the wake-up engine at the second moment being less than a second threshold, controlling the application processor to return from the ready state to the standby state.
4. The method according to claim 1, wherein: The method further comprises: In response to the application processor entering the active state from the ready state, the wake-up engine is controlled to enter a second working mode from a first working mode, and the power consumption of the second working mode is lower than that of the first working mode.
5. The method according to claim 1 or 2, wherein: In response to a second evaluation value of the electronic device for the audio data collected by the wake-up engine at a second moment being greater than or equal to a second threshold, controlling the application processor to enter an active state from the ready state includes: In response to the electronic device having a third evaluation value of the audio data collected by the wake-up engine at a third moment being greater than or equal to a second threshold, and the evaluation values from the third moment to the second moment being greater than the second threshold, controlling the application processor to enter an active state from the ready state, wherein the third moment is earlier than the second moment.
6. The method according to claim 5, wherein: The duration from the third moment to the second moment varies with the duration of the wake-up word.
7. The method according to claim 1, wherein: The method further comprises: Acquire the ambient audio collected by the wake-up engine in real time, wherein the ambient audio is an expression of the real environment; Extracting audio data from the ambient audio, wherein the audio data is voice information of a target user; and Calculate an evaluation value of the audio data at the current moment, wherein the evaluation value represents a degree of match between the audio data and the sound feature of the wake-up word.
8. The method according to claim 1, wherein: The method further comprises: In response to the application processor entering the active state, the electronic device is controlled to display a user interface and wake up a target application in the electronic device, where the target application is used to enable voice interaction between the target user and the electronic device.
9. A control device for an electronic device, comprising: A first control module, configured to control an application processor of the electronic device to enter a ready state from a standby state in response to a first evaluation value of the electronic device for the audio data collected by the wake-up engine at a first moment being greater than or equal to a first threshold; as well as a second control module, configured to control the application processor to enter an active state from the ready state in response to a second evaluation value of the electronic device for the audio data collected by the wake-up engine at a second moment being greater than or equal to a second threshold; Among them, the evaluation value represents the degree of match between the audio data collected by the wake-up engine of the electronic device and the wake-up word of the electronic device, the second threshold is greater than the first threshold, the power consumption of the electronic device in the standby state is less than that in the ready state, and the power consumption in the ready state is less than that in the active state.
10. An electronic device, comprising: Wake up the engine; A memory for storing one or more programs; One or more processors, when the one or more programs are executed, cause the wake-up engine and the one or more processors to execute the method according to any one of claims 1 to 8.
