Voice control method and device, and storage medium

Abstract

The invention provides a voice control method and device and a storage medium, which are applied to electronic equipment, and the method comprises the following steps: in response to an operation of a user to turn on a voice control switch, a service processing module subscribes a scene event of a target application firstly; when a scene event is started, the target application sends scene event data to the business processing module, and after the business processing module passes verification, the scene event data is sent to the scene awareness module; and the context awareness module triggers a fence of a scene event and informs the service processing module, and the service processing module pulls up the voice processing module to perform radio recognition after making a decision. According to the scheme, the equipment can automatically start radio recognition after the scene event occurs, so that the voice instruction of the user can be responded in time, and the voice control efficiency of the equipment is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a voice control method, device and storage medium. Background Art

[0002] With the continuous development of artificial intelligence technology, the functions of smart electronic devices are becoming more and more powerful. For example, users can control electronic devices through voice. At present, voice control is mainly achieved through the voice assistant of electronic devices. After the voice assistant is awakened, the electronic device can be voice controlled. For example, the user needs to first wake up the voice assistant through touch buttons, interface controls or say the wake-up word, and then say the voice command to perform voice control. The above voice control scheme relies on the voice assistant, which leads to problems such as extended time and low voice control efficiency. Summary of the invention

[0003] The embodiments of the present application provide a voice control method, device, and storage medium, which are applied to the field of terminal technology to achieve instant response to user voice commands and improve the voice control efficiency of the device.

[0004] In the first aspect, an embodiment of the present application proposes a voice control method, which is applied to an electronic device, wherein the electronic device includes a target application, a business processing module, a scenario perception module and a voice processing module, and the method includes: in response to a user turning on a voice control switch, the business processing module subscribes to a fence of a scenario event of the target application to the scenario perception module; when the scenario event of the target application is initiated, the target application sends scenario event data to the scenario perception module through the business processing module; if the scenario perception module determines that the scenario event has been subscribed based on the scenario event data, the scenario perception module notifies the business processing module, and the business processing module controls the voice processing module to be pulled up; and the voice processing module performs sound reception recognition.

[0005] In this embodiment, the target application is located in the application layer of the electronic device, and the target application can be a third-party application or a system application. The target application includes but is not limited to an alarm clock application, a phone application, etc. The business processing module, the context perception module, and the voice processing module are located in the application framework layer of the electronic device.

[0006] In the above scheme, after the user turns on the voice control switch, the business processing module first subscribes to the scene event of the target application. When the scene event is initiated, the target application sends the scene event data to the business processing module. After the business processing module verifies it, it sends the scene event data to the scenario perception module. When the scenario perception module triggers the fence of the scene event, it notifies the business processing module. After the business processing module makes a decision, it pulls up the voice processing module to perform sound reception recognition. It can be seen that after the scene event occurs, the device automatically turns on sound reception recognition, so that the device can respond to the user's voice commands immediately, improving the voice control efficiency of the device.

[0007] In an optional embodiment of the first aspect, when a scenario event of a target application is initiated, the target application sends scenario event data to the business processing module, including: when a scenario event of a target application is initiated, the target application sends scenario event data to the business processing module by calling a software program toolkit;

[0008] The scene event data includes the identifier of the target application and the identifier of the scene event.

[0009] In the above scheme, the target application can be a third-party application. The third-party application can integrate a software program toolkit to access the business processing module of the device, so that the third-party application can use the voice control capability provided by the equipment manufacturer to realize that the device automatically starts sound reception recognition after the scene event of the third-party application occurs, so that the device can respond instantly to the voice commands issued by the user to the third-party application, thereby expanding the scenarios and applications supported by the device.

[0010] In an optional embodiment of the first aspect, the target application sends scene event data to the scenario perception module through the business processing module, including: the target application sends the scene event data to the business processing module; the business processing module verifies the scene event data, and if the verification passes, the business processing module sends the scene event data to the scenario perception module.

[0011] In an optional embodiment of the first aspect, the business processing module verifies the scene event data. If the verification passes, the business processing module sends the scene event data to the situational perception module, including: the business processing module verifies the data format of the scene event data; if the data format of the scene event data meets the preset format requirements, the business processing module sends the scene event data to the situational perception module.

[0012] In the above scheme, the business processing module receives the scene event data of the target application and needs to verify the scene event data. After the verification, the data is transmitted to the situational awareness module to ensure the integrity and accuracy of the transmitted data.

[0013] In an optional embodiment of the first aspect, the electronic device further includes a service management module, a sound receiving module and a voice recognition module, and the service processing module controls the voice processing module to be started, including:

[0014] The business processing module sends scene event data to the voice processing module;

[0015] The voice processing module performs voice recognition, including:

[0016] The speech processing module queries the preset instruction words of the scene event based on the scene event data;

[0017] If the preset instruction word of the scene event is found, the voice processing module performs voice recognition by calling the sound receiving module and the voice recognition module to obtain the voice recognition result.

[0018] In this embodiment, the service management module, the sound receiving module and the voice recognition module are located in the application framework layer of the electronic device.

[0019] In the above scheme, after receiving the scene event data, the voice processing module first queries the validity of the scene event. If the preset instruction word of the scene event is found, the scene event is valid, and the voice processing module starts the sound reception recognition. The voice processing module can call the voice recognition module to obtain the voice recognition result.

[0020] In an optional embodiment of the first aspect, if the voice recognition result hits the preset instruction word of the scene event, the method also includes: the voice processing module sends a control instruction to the target application by calling the service management module; the target application executes the corresponding service operation based on the control instruction.

[0021] In the above scheme, if the voice command issued by the user is consistent with the preset command word of the scene event, the device will immediately respond to the voice command issued by the user and perform the corresponding service operation, thereby improving the efficiency of device voice control. For example, after the alarm rings, the user says "turn off the alarm", and after the voice processing module recognizes the command word, it controls the alarm application to turn off the alarm by calling the service management module.

[0022] In an optional embodiment of the first aspect, the method further includes: after the scene event of the target application ends, the target application sends a first message to the voice processing module by calling the service management module, and the first message is used to notify the end of the scene event; the voice processing module turns off the sound reception recognition based on the first message. Fig.10 Control response of the illustrated embodiment.

[0023] In the above solution, after the scene event of the target application ends, the service management module can be called to notify the voice processing module to turn off the sound reception to save device power consumption.

[0024] In an optional embodiment of the first aspect, the method also includes: after the situation awareness module learns that the situation event of the target application has ended, the situation awareness module sends a second message to the voice processing module through the business processing module, and the second message is used to notify the end of the situation event; the voice processing module turns off the sound reception recognition based on the second message.

[0025] In the above scheme, the business processing module pre-subscribes to the fence of the end of the scene event of the target application. After the situation awareness module triggers the fence, it notifies the business processing module, and the business processing module notifies the voice processing module to turn off the reception to save device power consumption.

[0026] In an optional embodiment of the first aspect, the method further includes: the voice processing module periodically obtains preset instruction words of the scene events of the target application from the server of the target application; and stores the preset instruction words of the scene events of the target application.

[0027] In the above scheme, the server of the target application provides preset command words for each scene event of the target application. It can be understood that as the target application business develops, more services are provided, the scene events are richer, and the preset command words will be more and more accurate. Therefore, the voice processing module needs to periodically interact with the target application server to obtain the latest preset command words for scene events to enhance the user's voice control experience.

[0028] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the method described in any one of the first aspects.

[0029] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the first aspects is implemented.

[0030] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the method described in any one of the first aspects.

[0031] In a fifth aspect, an embodiment of the present application provides a chip system, which includes at least one processor and at least one interface circuit, the interface circuit is used to receive or send a signal, and the processor is used to call a computer program in a memory to execute a method as described in any one of the first aspects.

[0032] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of an interface change provided in an embodiment of the present application;

[0034] Figure 2 A schematic diagram of an interface change provided in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of an interface change provided in an embodiment of the present application;

[0036] Figure 4A schematic diagram of an interface provided in an embodiment of the present application;

[0037] Figure 5 A schematic diagram of an interface provided in an embodiment of the present application;

[0038] Figure 6 A schematic diagram of an interface provided in an embodiment of the present application;

[0039] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0040] Figure 8 A software structure block diagram of an electronic device provided in an embodiment of the present application;

[0041] Fig. 9 A flow chart of a voice control method provided in an embodiment of the present application;

[0042] Fig.10 A flow chart of a voice control method provided in an embodiment of the present application;

[0043] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0044] Fig.12 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0046] Voice assistant: is the name of a voice control function. Users can turn on or off the voice assistant by turning on or off the preset switch on the electronic device. After the voice assistant function is turned on, the user needs to say the wake-up word first, and then say the voice command after the voice assistant of the electronic device is awakened to control the electronic device to perform the corresponding service operation.

[0047] Scenario-based commands: This is the name of another voice control function. Users can turn on or off the scenario-based command function by turning on or off the preset switch on the electronic device. Unlike the voice assistant and see-and-speak functions, after the scenario-based command function is turned on, the electronic device will only enter a short reception state when a specific scene (for example, an alarm rings) is triggered. The user directly speaks a voice command to control the electronic device to perform the corresponding service operation (for example, turn off the alarm). After the specific scene ends (for example, the alarm turns off), the electronic device turns off the reception.

[0048] Fence: In the embodiment of the present application, the fence mainly refers to the fence of the scene event, such as the fence of the alarm scene event, the fence of the phone scene event, etc. The context awareness module of the electronic device is used to create a fence, register a fence, and monitor fence events.

[0049] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0050] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0051] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to voice data used for analysis, storage, and display, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0052] In some embodiments, the user can turn on the voice assistant function by turning on the "voice wake-up" switch on the electronic device. For example, if the electronic device is a mobile phone, Figure 1As shown, the user clicks the icon of the "Settings" application on the mobile phone desktop. In response to the user's click operation on the icon of the "Settings" application, the mobile phone displays the "Settings" interface 101. The "Settings" interface 101 includes a "Smart Voice" option 1011, and the "Smart Voice" option 1011 is used to set the smart voice function. In one example, in response to the user's click operation on the "Smart Voice" option 1011, the mobile phone displays the "Smart Voice" interface 102, and the "Smart Voice" interface 102 includes a "Voice Wake-up" option 1021. The user can click the "Voice Wake-up" option 1021 to set options related to the voice wake-up function. In one example, in response to the user's click operation on the "Voice Wake-up" option 1021, the mobile phone displays the "Voice Wake-up" interface 103. Optionally, the "Voice Wake-up" interface 103 includes a "Voice Wake-up" switch 1031, an option 1032 for setting a wake-up word, and an option 1033 for customizing a response word. The mobile phone defaults to the "Voice Wake-up" switch 1031 being off. In one example, the user can click on the "voice wake-up" switch 1031. In response to the user's click on the "voice wake-up" switch 1031, the mobile phone displays the "voice wake-up" interface 104, and the "voice wake-up" switch 1041 of the "voice wake-up" interface 104 is turned on, and the mobile phone turns on the voice assistant function. In addition, the user can select the default wake-up word (for example, "Hello, YOYO") or a custom wake-up word in the option 1032 for setting the wake-up word, and can also click "Re-enter the wake-up word" to update the custom wake-up word. The user can also click on the option 1033 of "Customized answer language" to enter a customized question and answer language.

[0053] After the voice assistant function is turned on, the user can wake up the voice assistant of the electronic device by saying the wake-up word. After the voice assistant is awakened, the user can say a voice command to control the electronic device to perform the corresponding service operation. However, there are problems such as low wake-up efficiency, poor voice control experience caused by wake-up difficulties, and time delay caused by reliance on the voice assistant. In order to improve the efficiency of device voice control and respond immediately to the user's voice command for a certain application event, the embodiment of the present application proposes a scenario-based command function.

[0054] The following is a detailed description of the scenario-based command function.

[0055] In some embodiments, the user can enable the scenario-based instruction function by turning on the "scenario-based instruction" switch on the electronic device. For example, the electronic device is a mobile phone. Figure 2As shown, the user clicks the icon of the "Settings" application on the mobile phone desktop. In response to the user's click operation on the icon of the "Settings" application, the mobile phone displays the "Settings" interface 201. The "Settings" interface 201 includes a "Smart Assistant" option 2011, and the "Smart Assistant" option 2011 is used to set related functions of smart services and smart interactions. In one example, in response to the user's click operation on the "Smart Assistant" option 2011, the mobile phone displays the "Smart Assistant" interface 202, and the "Smart Assistant" interface 202 includes a "Voice Control" option 2021. The user can click the "Voice Control" option 2021 to set options related to the voice control function. In one example, in response to the user's click operation on the "Voice Control" option 2021, the mobile phone displays the "Voice Control" interface 203, and the "Voice Control" interface 203 includes a "Scenario-based Instructions" switch 2031, and the mobile phone defaults to the "Scenario-based Instructions" switch 2031 as off. In one example, a user can click on the "scenario-based instruction" switch 2031. In response to the user's click on the "scenario-based instruction" switch 2031, the mobile phone displays the "voice control" interface 204. The "scenario-based instruction" switch 2041 of the "voice control" interface 204 is on, and the mobile phone turns on the scenario-based instruction function.

[0056] After the scenario-based command function is turned on, when an event in a specific scenario is triggered, the electronic device enters a short reception state, and the electronic device will turn off the reception after the event in the specific scenario ends. For example, when the alarm rings or the phone rings, the electronic device enters a short reception state, and the microphone of the electronic device will turn off the reception after the alarm turns off or the phone is answered.

[0057] In one example, a user can manage scenes on an electronic device. For example, if the electronic device is a mobile phone, Figure 3As shown, the "voice control" interface 301 includes a "scenario-based instruction" option 3011, and the "scenario-based instruction" option 3011 is used to set the related functions of the scenario-based instruction. In one example, in response to the user's click operation on the "scenario-based instruction" option 3011, the mobile phone displays the "scenario-based instruction" interface 302, and the "scenario-based instruction" interface 302 includes a voice control switch 3022 of the alarm clock application and a voice control switch 3023 of the phone application. Taking the alarm clock scene as an example, the mobile phone defaults to the voice control switch 3022 of the alarm clock application as off. In response to the user's click operation on the voice control switch 3022 of the alarm clock application, the mobile phone displays the "scenario-based instruction" interface 304, and the voice control switch 3041 of the alarm clock application in the "scenario-based instruction" interface 304 is on, and the mobile phone turns on the voice instruction function of the alarm clock scene. Optionally, the "scenario-based instruction" interface 302 also includes an icon 3021, and the user can click the icon 3021 to view the usage method of the voice control function in the alarm clock scene. In one example, in response to the user clicking on the icon 3021, the mobile phone displays the "scenario-based instruction" interface 303, and the "scenario-based instruction" interface 303 displays a prompt box 3031, which includes prompt information, and the prompt information is used to prompt the user how to use the voice control function in the alarm clock scenario. In addition, the user clicks the "Got it" control 3032 in the prompt box 3031, and the mobile phone closes the prompt box 3031. The activation of the voice control function in the phone scenario is similar to the activation of the voice control function in the alarm clock scenario. Please refer to the relevant description of the alarm clock scenario, which will not be expanded here.

[0058] Optionally, switches for more applications may be expanded in the "scenario-based instructions" to enable voice command functions for scenarios corresponding to more applications, which is not limited to the embodiments of the present application.

[0059] After the switch of the scenario-based command function is turned on, when an event in a specific scenario is triggered, the electronic device turns on the sound reception. In response to receiving the voice input by the user, the electronic device recognizes the voice and executes the command corresponding to the user's voice. After the command is executed, the sound reception is turned off.

[0060] For example, Figure 4 As shown, after the switch of the scenario-based command function is turned on, when the alarm rings, the mobile phone displays a lock screen interface 401, and a prompt message 4011 is displayed on the lock screen interface 401. The user can say the voice command "turn off the alarm" according to the prompt message, and the mobile phone executes the operation of turning off the alarm.

[0061] For example, Figure 5As shown, after the switch of the scenario-based command function is turned on, when the alarm rings, the mobile phone displays the main interface 501, a prompt window 5012 pops up above the main interface 501, and a prompt message 5011 is displayed below the prompt window 5012. The user can say the voice command "turn off the alarm" according to the prompt message, and the mobile phone will execute the operation of turning off the alarm.

[0062] For example, Figure 6 As shown, after the switch of the scenario-based command function is turned on, the mobile phone displays a short video playback interface 601, and the short video playback interface 601 is playing a short video. When the alarm rings, the mobile phone pauses playing the short video. In one example, a "play" button 6013 is displayed on the short video playback interface 601 to prompt the user that the short video has been paused. At the same time, a prompt window 6012 will pop up above the short video playback interface 601, and a prompt message 6011 will be displayed below the prompt window 6012. The user can say the voice command "turn off the alarm" according to the prompt message, and the mobile phone will execute the operation of turning off the alarm.

[0063] In one example, after the switch of the scenario-based command function is turned on, when an event in a specific scenario is triggered, the electronic device enters a short receiving state, and a prompt icon is displayed on the display interface of the electronic device. The prompt icon is used to prompt that the microphone is turned on and the electronic device is in the receiving state, thereby preventing the user's privacy from being leaked.

[0064] For example, after the scene-based command function is turned on, when the alarm rings, the phone displays Figure 4 In the lock screen interface 401 shown in the figure, the status bar of the lock screen interface 401 displays a prompt icon 4012, which is used to prompt that the microphone is turned on. The user speaks a voice command such as "turn off the alarm clock". After the mobile phone executes the operation of turning off the alarm clock, the prompt icon 4012 is no longer displayed, and the mobile phone turns off the radio. For another example, after the switch of the scene-based command function is turned on, when the alarm clock rings, the mobile phone displays Figure 5 The main interface 501 shown in the figure has a status bar 5013, and the prompt icon 5013 is used to prompt that the microphone is turned on. For another example, after the switch of the scene-based command function is turned on, when the alarm rings, the mobile phone displays Figure 6 The short video playback interface 601 shown, the status bar of the short video playback interface 601 displays 6014, and the prompt icon 6014 is used to prompt that the microphone is turned on.

[0065] An embodiment of the present application provides a voice control method. After the switch of the scenario-based command function is turned on, the electronic device subscribes to the event fence of the specific scenario supported by the scenario-based command function. When a specific scenario (for example, the alarm rings) is triggered, the electronic device enters a short radio state, and the microphone of the electronic device continuously collects surrounding sounds for a period of time. At this time, the user does not need to say the wake-up word, and can control the electronic device to perform service operations for specific scenarios through voice commands, which can improve the efficiency of voice control. After the electronic device performs a service operation (for example, turning off the alarm or delaying the alarm), turn off the radio, which can save device power consumption.

[0066] It should be noted that the voice control solution provided in the embodiment of the present application can be executed on the terminal side (electronic device side), and the terminal side does not need to be connected to the network side, that is, the terminal side can execute the above-mentioned voice control solution in an offline state.

[0067] The voice control method provided in the embodiment of the present application can be applied to any electronic device with a voice control function, and the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile Internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a smart screen, an artificial intelligence (AI) speaker, a headset, a terminal in industrial control, a terminal in self-driving, a terminal in remote medical surgery, a terminal in a smart grid, a terminal in transportation safety, a terminal in a smart city, a terminal in a smart home, a personal digital assistant (PDA), etc., and the embodiment of the present application does not make any limitation to this.

[0068] For example, Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 7As shown, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0069] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, a display processing unit (displayprocess unit, DPU), and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. In some embodiments, the electronic device 100 may also include one or more processors 110. Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. A memory may also be set in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a high-speed cache memory. The memory may save instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the electronic device 100 system.

[0070] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. Among them, the USB interface 130 is an interface that complies with the USB standard specification, and can specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect headphones and play audio through the headphones.

[0071] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0072] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device 100 through the power management module 141.

[0073] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0074] The wireless communication function of the electronic device 100 can be implemented by antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, antenna 1 can be reused as a diversity antenna of a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0075] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier, etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0076] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0077] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN), Bluetooth, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of the signal, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0078] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include GSM, GPRS, CDMA, WCDMA, TD-SCDMA, LTE, GNSS, WLAN, NFC, FM, and / or IR technology. The above-mentioned GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0079] The electronic device 100 can realize the display function through the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute instructions to generate or change display information.

[0080] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0081] The electronic device 100 can implement a shooting function through an ISP, one or more cameras 193, a video codec, a GPU, one or more display screens 194, and an application processor.

[0082] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0083] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, data files such as music, photos, and videos are stored in the external memory card.

[0084] The internal memory 121 can be used to store one or more computer programs, which include instructions. The processor 110 can enable the electronic device 100 to perform various functional applications and data processing by running the above instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system; the program storage area can also store one or more applications (such as a gallery, contacts, etc.). The data storage area can store data (such as photos, contacts, etc.) created during the use of the electronic device 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. In some embodiments, the processor 110 can enable the electronic device 100 to perform various functional applications and data processing by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor 110.

[0085] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0086] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0087] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0088] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0089] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0090] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130, a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0091] The sensor 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

[0092] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0093] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting or removing the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

[0094] The software system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system as an example to illustrate the software structure of the electronic device. Figure 8 A software structure block diagram of an electronic device provided in an embodiment of the present application. The layered architecture divides the software system of the electronic device into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into four layers, namely, the application layer (applications), the application framework layer (application framework), the Android runtime (Android runtime) and the system library, and the kernel layer (kernel).

[0095] The application layer may include a series of application packages. In the embodiment of the present application, Figure 8 As shown, the application layer includes applications such as alarm clock, camera, phone, video, music, gallery, calendar, map, navigation, Bluetooth, short message, etc. The application can be a system application or a third-party application.

[0096] The application framework layer provides an application programming interface (API) and a programming framework for the application programs in the application layer. Figure 8 As shown, the application framework layer includes a business processing module, a context awareness module, a service management module, a speech processing module, an echo cancellation module, and a speech recognition module.

[0097] The service processing module is used to verify the scene event data, and can make multi-modal decisions based on the current device state and scene event data, and determine whether to pull up the voice processing module based on the decision results, where the device state includes but is not limited to the device display state, motion state, folding screen state, application running state, etc. In other embodiments, the service processing module can also pull up gesture services or eye movement services based on the decision results.

[0098] The context awareness module is used to form an event fence according to the scene events sent by the application. The service management module is used to perform corresponding service operations on the application according to the instructions transmitted by the voice processing module.

[0099] The voice processing module is used for voice control in the short receiving state, and the voice processing module can also be called a scenario-based command voice processing module. After the switch of the scenario-based command function is turned on, when an event in a specific scene is triggered, the electronic device pulls up the voice processing module, the voice processing module performs voice reception recognition, and performs voice control. After the event in the specific scene ends, the voice processing module stops receiving voice recognition. In one example, after the switch of the scenario-based command function is turned on, when the alarm rings, the electronic device pulls up the voice processing module, the voice processing module turns on voice reception recognition and performs voice control, when the user says "turn off the alarm" or "remind later", the alarm can be turned off or delayed, after turning off or delaying the alarm, the voice processing module turns off the radio. In one example, after the switch of the scenario-based command function is turned on, when the phone rings, the electronic device pulls up the voice processing module, the voice processing module turns on voice reception recognition and performs voice control, when the user says "answer the call" or "hang up the call", the call can be answered or hung up, after answering or hanging up the call, the voice processing module turns off the radio. Both of the above examples do not require a wake-up operation in advance.

[0100] The voice processing module is also used to regularly maintain and manage preset command words for specific scenarios, etc. The voice processing module can regularly check the validity of the preset command words for specific scenarios cached locally, and update the preset command words for specific scenarios by interacting with the cloud.

[0101] The echo cancellation module is used to eliminate noise and echo in the audio signal to improve the audio clarity. The speech processing module can obtain a clear audio signal by calling the echo cancellation module.

[0102] The speech recognition module provides speech recognition capability for recognizing the user's speech command. In some embodiments, the speech recognition module may also be referred to as an intelligence capability module. The speech processing module may obtain the speech recognition result of the audio signal by calling the speech recognition module.

[0103] In some embodiments, the application framework layer may include an input manager service (IMS). Of course, the application framework layer may also include a display policy service, a power management service (PMS), a display management service (DMS), an activity manager, a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc., which are not limited in the embodiments of the present application.

[0104] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0105] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0106] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0107] The system library can include multiple functional modules. For example: state monitoring service, surface manager, media library, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc. The state monitoring service is used to determine the specific orientation of the mobile phone, the physical state of the flexible screen, etc. according to the monitoring data reported by the kernel layer. The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0108] The kernel layer is a layer between hardware and software. The kernel layer at least includes display driver, camera driver, audio driver, sensor driver, etc., and the present application embodiment does not impose any restrictions on this.

[0109] The following specific embodiments are used to describe the technical solution of the present application in detail. The following specific embodiments can be implemented independently or in combination with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0110] For example, Fig. 9 The flow chart of a voice control method provided in an embodiment of the present application is shown in FIG. The voice control method can be applied to electronic devices with voice control functions, such as Fig. 9 As shown, the electronic equipment includes:

[0111] Third-party applications, business processing module, scenario awareness module, voice processing module, echo cancellation module, voice recognition module and service management module. Among them, the business processing module includes the permission verification module, the multi-mode decision module and the service startup module. The scenario awareness module includes the event fence module, the voice processing module includes the voice control module, the scene management module, the sound receiving module and the command recognition module. The service management module includes the service execution module.

[0112] Based on the electronic device, the voice control method of this embodiment includes the following processes:

[0113] 1) Scene subscription

[0114] Step 1-1: The business processing module subscribes to the fence of the scene event in the scenario perception module. Specifically, the multi-mode decision module of the business processing module subscribes to the fence of the scene event from the scenario perception module. For example, when it is detected that the user turns on the voice control switch corresponding to the alarm scene event, the business processing module subscribes to the event fence of the alarm scene from the scenario perception module. For another example, when it is detected that the user turns on the voice control switch corresponding to the phone scene event, the business processing module subscribes to the event fence of the phone scene from the scenario perception module.

[0115] 2) Business processing

[0116] Step 2-1: The third-party application sends the scene event data to the business processing module. Specifically, the third-party application sends the scene event data to the permission verification module of the business processing module by calling a software development kit (SDK).

[0117] Step 2-2: The business processing module verifies the scene event data, and after passing the verification, sends the scene event data to the context perception module. Specifically, the permission verification module of the business processing module verifies the data format of the received scene event data. If it meets the preset format requirements of the scene event data, the permission verification module sends the scene event data to the event fence module of the context perception module.

[0118] Step 2-3: After the context perception module determines the fence that triggers the context event based on the context event data, it sends the context event data to the business processing module. Specifically, after the event fence module of the context perception module determines the fence that triggers the context event, it sends the context event data to the multi-mode decision module of the business processing module.

[0119] Step 2-4: The business processing module makes a decision based on the mobile phone status and scene event data to determine whether to start the voice service. Specifically, the multi-mode decision module of the business processing module makes a decision based on the mobile phone status and scene event data to determine whether to start the voice service.

[0120] Step 2-5: The service processing module controls the start-up of the voice processing module. Specifically, the multi-mode decision module of the service processing module sends an instruction to the service start-up module so that the service start-up module notifies the start-up of the voice processing module.

[0121] 3) Voice Processing

[0122] Step 3-1: After the voice processing module is started, the validity of the scene is verified. Specifically, after the voice control module of the voice processing module is started, the voice control module notifies the scene management module to verify the validity of the scene.

[0123] Step 3-2: The voice processing module inquires that the scene exists and starts the sound receiving module in the voice processing module to receive sound. Specifically, the voice control module of the voice processing module learns from the scene management module that the scene exists (ie, the scene is a valid scene), and the voice control module notifies the sound receiving module to start receiving sound.

[0124] Among them, the scene management module stores preset command words for different scenes, that is, command words preset by third-party applications. It is understandable that by interacting with the cloud server, the scene management module can obtain preset command words for different scenes. If the electronic device locally stores preset command words for a certain scene, it can be determined that the scene is a valid scene.

[0125] Exemplarily, the scene management module may periodically obtain preset command words for different scenarios from the cloud server. For example, the scene management module may periodically obtain preset command words for scene events of a third-party application from the cloud server of the third-party application, where the third-party application may be a video application, an audio application, a camera application, etc.

[0126] Step 3-3: The sound receiving module calls the echo cancellation module to receive sound.

[0127] Step 3-4: The audio receiving module receives the voice command issued by the user.

[0128] Step 3-5: The sound receiving module sends the voice command after echo cancellation to the command recognition module in the voice processing module.

[0129] Step 3-6: The command recognition module calls the voice recognition module in the intelligent capability to perform scene command recognition. Specifically, the voice recognition module sends the recognized voice command to the command recognition module, and the command recognition module determines whether the recognized voice command is consistent with the preset command word of the current scene. If the voice command is consistent with the preset command word, the preset command word is included in the voice recognition result; if the voice command is inconsistent with the preset command word, the sound reception recognition continues.

[0130] Step 3-7: The instruction recognition module sends the speech recognition result to the service management module. Specifically, the instruction recognition module sends the speech recognition result to the service execution module in the service management module.

[0131] 4) Service Execution

[0132] Step 4-1: The service management module sends a control instruction to the third-party application, and the control instruction is used to instruct the third-party application to perform a corresponding service operation. Specifically, if the speech recognition result received by the service execution module of the service management module carries a preset instruction word, the service execution module sends a control instruction to the third-party application to instruct the third-party application to perform the service operation corresponding to the preset instruction word.

[0133] The voice control method shown in this embodiment involves the process of event subscription, business processing, voice processing and service instructions. Based on this process, the electronic device can avoid waking up the voice assistant and respond to the voice instructions issued by the user immediately, realizing efficient and convenient voice interaction and improving the user's voice control experience. The above method shows that a third-party application can access the device provided by the integrated SDK. Fig. 9 The voice control framework shown, that is, the third-party application calls the interface provided by the voice control framework and uses the above voice control method, so that when the user uses the third-party application, when the scene event registered by the third-party application is triggered, the user can directly speak a voice command to control the mobile phone to perform the corresponding service operation, and efficiently and quickly complete the control operation of the third-party application event, thereby improving the user experience. Application developers can configure commonly used voice command words for the application based on the voice control framework, and the device can periodically obtain the latest voice command words from the application server so that the device can match the command words when it is offline.

[0134] For example, Fig.10 A flow chart of a voice control method provided in an embodiment of the present application. Fig.10 As shown, the voice control method can be applied to any electronic device with a voice control function, and the method includes:

[0135] S1001. In response to the user turning on the voice control switch, the business processing module subscribes to the fence of the target scene event from the scenario awareness module.

[0136] The voice control switch is the control switch of the voice processing module. The voice processing module is used for voice control in the short reception state. The voice processing module can also be called a scenario-based command voice processing module. Turning on the voice control switch does not mean directly pulling up the voice processing module. When the target scene event is triggered, the electronic device pulls up the voice processing module, the voice processing module performs reception recognition, and performs voice control. Among them, the target scene event includes at least one of an alarm scene event (alarm ringing) and a phone scene event (phone ringing).

[0137] In an optional embodiment, the voice control switch is Figure 2 The “scenario-based instruction” switch 2031 in the “voice control” interface 203 shown, in response to the user turning on the “scenario-based instruction” switch 2031, the business processing module subscribes to the fences of the alarm scenario event and the phone scenario event to the context awareness module.

[0138] In an optional embodiment, the voice control switch is Figure 3 The voice control switch 3022 of the alarm application in the "scenario-based instruction" interface 302 is shown. In response to the user turning on the voice control switch 3022 of the alarm application, the business processing module subscribes to the fence of the alarm scenario event to the scenario perception module.

[0139] In an optional embodiment, the voice control switch is Figure 3 The voice control switch 3023 of the phone application in the "scenario-based instruction" interface 302 is shown. In response to the user turning on the voice control switch 3023 of the phone application, the business processing module subscribes to the fence of the phone scenario event from the context awareness module.

[0140] The business processing module subscribes to the fence of the target scene event from the context perception module, and the context perception module registers the fence of the target scene event, such as registering the fence of the alarm scene event and registering the fence of the phone scene event. In some embodiments, the context perception module can also register the motion state fence of the electronic device, the screen folding state fence, etc.

[0141] S1002. When the target scene event is initiated, the target application sends the scene event data to the business processing module.

[0142] In an optional embodiment, the scene event data includes an identifier of a target application and an identifier of a target scene event.

[0143] Exemplarily, the target scene event is an alarm scene event, such as an alarm ringing, and the target application is an alarm application. Exemplarily, when the alarm rings, the alarm application sends alarm scene event data to the service processing module, and the alarm scene event data includes an identifier of the alarm application and an identifier of the alarm ringing event.

[0144] Exemplarily, the target scenario event is a telephone scenario event, such as a phone ringing, and the target application is a phone application. Exemplarily, when the phone rings, the phone application sends phone scenario event data to the service processing module, and the phone scenario event data includes an identifier of the phone application and an identifier of the phone ringing event.

[0145] S1003. The business processing module performs scene event verification based on the scene event data.

[0146] After the scene event verification passes, execute S1004, otherwise, end this process.

[0147] In an optional embodiment, the scene event verification includes format verification of the scene event data, etc. If the format of the scene event data meets the preset format requirement, S1004 is executed; or if the format of the scene event data does not meet the preset format requirement, this process ends.

[0148] S1004. The business processing module sends scene event data to the situation awareness module.

[0149] The business processing module stores preset data format requirements for different scene event data. The business processing module verifies the data format of the scene event data based on the preset data format requirements to ensure that the transmitted scene event data meets the data format requirements.

[0150] In an optional embodiment, the scene event data received by the business processing module is alarm scene event data. By querying the preset data format of the alarm scene event, it is verified whether the alarm scene event data meets the preset data format of the alarm scene event. If so, the business processing module sends the alarm scene event data to the situation perception module.

[0151] In an optional embodiment, the scene event data received by the business processing module is telephone scene event data. By querying the preset data format of the telephone scene event, it is verified whether the telephone scene event data meets the preset data format of the telephone scene event. If so, the business processing module sends the telephone scene event data to the scenario perception module.

[0152] S1005. The situational awareness module determines whether to trigger the fence of the target scene event based on the scene event data.

[0153] If the situational awareness module determines that the fence triggers the target scene event, S1006 is executed, otherwise the process ends.

[0154] S1006. The context awareness module sends a subscription response to the business processing module.

[0155] The context perception module receives the scene event data, determines whether the business processing module subscribes to the fence of the target scene event, and if it is determined that the business processing module has subscribed to the fence of the target scene event, the context perception module determines the fence that triggers the target scene event, and the context perception module sends a subscription response to the business processing module, and the subscription response is used to notify the business processing module that the target scene event is started. Optionally, the subscription response includes the scene event data, and the scene event data may include the identifier of the target application and the identifier of the target scene event.

[0156] In an optional embodiment, the scene event data received by the scenario perception module is alarm scene event data, and the scenario perception module determines whether the business processing module has subscribed to the fence of the alarm scene event. If it is determined that the business processing module has subscribed to the fence of the alarm scene event, the scenario perception module sends a subscription response to the business processing module, and the subscription response is used to notify the business processing module that the alarm scene event is started, wherein the subscription response may include an identifier of the alarm application and an identifier of the alarm ringing event.

[0157] In an optional embodiment, the scenario event data received by the scenario perception module is telephone scenario event data, and the scenario perception module determines whether the business processing module has subscribed to the fence of the telephone scenario event. If it is determined that the business processing module has subscribed to the fence of the telephone scenario event, the scenario perception module sends a subscription response to the business processing module, and the subscription response is used to notify the business processing module that the telephone scenario event is initiated, wherein the subscription response may include an identifier of the telephone application and an identifier of the telephone ringing event.

[0158] S1007. The business processing module makes a multi-mode decision to determine whether to pull up the voice processing module.

[0159] If the business processing module determines to pull up the voice processing module, S1008 is executed; otherwise, this process ends.

[0160] In an optional embodiment, the business processing module can make a multi-mode decision based on the scene event data and the mobile phone status, and determine whether to pull up the voice processing module based on the decision result. The voice processing module is used for voice control in the short reception state, or in other words, the voice processing module performs reception recognition in the time period from the start to the end of the target scene event. The voice processing module can also be called a scenario-based command voice processing module. The mobile phone status includes but is not limited to the mobile phone display status, motion status, folding screen status, application running status, etc.

[0161] In one example, the business processing module pre-stores the identifiers of the scene events supported by the voice processing module. The business processing module can obtain the identifier of the target scene event from the scene event data, and based on the identifier of the target scene event, when determining that the target scene event is a scene event supported by the voice processing module, pull up the voice processing module.

[0162] In one example, the business processing module pre-stores the identifiers of applications supported by the voice processing module. The business processing module can obtain the identifier of the target application from the scene event data, and based on the identifier of the target application, when determining that the target application is an application supported by the voice processing module, pull up the voice processing module.

[0163] Exemplarily, the target scene event is an alarm scene event, such as an alarm ringing. When the alarm rings, the alarm application sends the alarm scene event data to the business processing module by calling the software development kit (SDK). The business processing module verifies the alarm scene event data. If the verification passes, the business processing module sends the alarm scene event data to the scenario perception module. The scenario perception module determines whether the business processing module subscribes to the fence of the alarm scene event based on the alarm scene event data. The scenario perception module determines that the business processing module has subscribed to the fence of the alarm scene event based on the alarm scene event data, and the scenario perception module sends a subscription response to the business processing module, and the subscription response is used to notify the business processing module that the alarm scene event is started. Optionally, the subscription response includes an alarm application identifier and an alarm scene event identifier. The business processing module determines to pull up the voice processing module based on the alarm application identifier / alarm scene event identifier in the subscription response.

[0164] Exemplarily, the target scenario event is a telephone scenario event, such as a ringing phone. When the phone rings, the telephone application sends the telephone scenario event data to the business processing module by calling the SDK. The business processing module verifies the telephone scenario event data. If the verification passes, the business processing module sends the telephone scenario event data to the scenario perception module. The scenario perception module determines whether the business processing module subscribes to the fence of the telephone scenario event based on the telephone scenario event data. The scenario perception module determines that the business processing module has subscribed to the fence of the telephone scenario event based on the telephone scenario event data, and the scenario perception module sends a subscription response to the business processing module, and the subscription response is used to notify the business processing module that the telephone scenario event is started. Optionally, the subscription response includes the identifier of the telephone application and the identifier of the telephone scenario event. The business processing module determines to pull up the voice processing module based on the identifier of the telephone application / identifier of the telephone scenario event in the subscription response.

[0165] S1008. The business processing module sends a notification message to the voice processing module, where the notification message is used to notify the voice processing module to start.

[0166] Among them, the notification message may include scene event data.

[0167] S1009. The voice processing module starts voice recognition according to the notification message.

[0168] S1010. When a preset instruction word of a target scenario event is identified, the voice processing module sends a control message to the service management module, where the control message includes an identifier of the target application and a control instruction corresponding to the preset instruction word.

[0169] S1011. The service management module sends a control instruction to the target application according to the control message.

[0170] S1012. The target application executes corresponding service operations according to the control instructions.

[0171] In an optional embodiment, the voice processing module obtains the voice input by the user by calling the sound receiving module, and obtains the voice recognition text by calling the voice recognition module to understand and recognize the voice input by the user. Optionally, the voice processing module obtains the user voice processed by the echo cancellation module by calling the sound receiving module and the echo cancellation module to improve the voice clarity.

[0172] In an optional embodiment, the voice processing module obtains the preset instruction words of the target scene event based on the scene event data in the notification message, and compares the voice recognition text with the preset instruction words of the target scene event. If the voice recognition text matches any one of the preset instruction words of the target scene event, the voice processing module sends a control instruction to the target application through the service management module, so that the target application performs the corresponding service operation according to the control instruction.

[0173] It should be noted that the matching of the speech recognition text and the preset instruction words of the target scene event can be that the two have the same semantics or are close in semantics, or the speech recognition text contains the preset instruction words of the target scene event, etc., and it is not necessary for the speech recognition text to be exactly the same as the preset instruction words of the target scene event. Figure 5 In the scenario of the alarm ringing, the user inputs the voice "turn off the alarm", and the preset instruction words for this scenario include "turn off the alarm". "Turn off the alarm" and "turn off the alarm" have the same semantics, that is, the two match.

[0174] In one example, the voice processing module pre-stores the correspondence between scene events and preset command words, and the voice processing module can periodically obtain the correspondence between scene events and preset command words from the cloud server. Exemplarily, the correspondence between scene events and preset command words is shown in Table 1.

[0175] Table 1

[0176] Scene Events Preset command words Alarm Clock Turn off alarm, snooze alarm, etc. Phone Ringing Answer the call, hang up, mute, etc.

[0177] In another example, the voice processing module pre-stores the correspondence between the application and the preset command word, and the voice processing module can periodically obtain the correspondence between the application and the preset command word from the cloud server. Exemplarily, the correspondence between the application and the preset command word is shown in Table 2.

[0178] Table 2

[0179] Application Name Preset command words Alarm Clock App Turn off alarm, snooze alarm, etc. Phone App Answer a call, hang up, mute, etc.

[0180] In some embodiments, after the target application performs the corresponding service operation according to the control instruction, the following may also be included:

[0181] S1013. The target application sends a control response to the service management module, where the control response is used to indicate that the service operation of the target application has been completed, wherein the control response includes an identifier of the target application.

[0182] S1014. The service management module sends a control response to the voice processing module according to the control response.

[0183] The service management module determines to send a control response to the voice processing module according to the identifier of the target application in the control response.

[0184] S1015. After receiving the control response, the voice processing module turns off the voice recognition.

[0185] For example, the target scene event is an alarm scene event. After the alarm application is closed or the alarm is delayed, the voice processing module turns off the sound reception recognition to save device power consumption. If the alarm is delayed, when the alarm rings again, S1002 to S1015 are re-executed.

[0186] For example, the target scenario event is a phone scenario event, and after the phone application hangs up the call, the voice processing module turns off the voice recognition to save device power consumption. If the phone rings again, S1002 to S1015 are executed again.

[0187] In other embodiments, after the target application performs the corresponding service operation according to the control instruction, the following may also be included: after the context perception module learns that the scene event of the target application has ended, the second message is sent to the voice processing module through the business processing module to trigger the voice processing module to turn off the sound reception recognition. In this embodiment, the business processing module pre-subscribes to the fence of the end of the scene event of the target application.

[0188] The above embodiment shows a voice control method, in response to the user's operation of turning on the voice control switch, the business processing module subscribes to the fence of the target scene event to the scenario perception module; when the target scene event is initiated, the business processing module first verifies the scene event data, and after the verification is passed, the business processing module sends the scene event data to the scenario perception module, and the scenario perception module determines whether to trigger the fence of the target scene event; after the fence of the target scene event is triggered, the business processing module decides to pull up the voice processing module, and the voice processing module starts the sound reception recognition, which can realize instant response to the user's voice command when the target scene event occurs. The user does not need to wake up the device first and then speak the voice command, thereby improving the voice control efficiency and the user's voice control experience.

[0189] Based on the above-mentioned embodiments, an embodiment of the present application provides a voice control method, which is applied to an electronic device, wherein the electronic device includes a target application, a business processing module, a scenario perception module and a voice processing module, and the method includes: in response to a user turning on a voice control switch, the business processing module subscribes to a fence of a scenario event of the target application to the scenario perception module; when the scenario event of the target application is started, the target application sends scenario event data to the scenario perception module through the business processing module; if the scenario perception module determines that the scenario event has been subscribed based on the scenario event data, the scenario perception module notifies the business processing module, and the business processing module controls the voice processing module to be pulled up; and the voice processing module performs sound reception recognition.

[0190] The target application is located in the application layer of the electronic device, and the target application can be a third-party application or a system application. The target application includes but is not limited to an alarm clock application, a phone application, etc. The business processing module, the context perception module, and the voice processing module are located in the application framework layer of the electronic device.

[0191] In this embodiment, after the user turns on the voice control switch, the business processing module first subscribes to the scene event of the target application. When the scene event is initiated, the target application sends the scene event data to the business processing module. After the business processing module verifies it, it sends the scene event data to the scene perception module. When the scene perception module triggers the fence of the scene event, it notifies the business processing module. After the business processing module makes a decision, it pulls up the voice processing module to perform sound reception recognition. It can be seen that after the scene event occurs, the device automatically turns on sound reception recognition, so that the device can respond to the user's voice command immediately, improving the voice control efficiency of the device.

[0192] In an optional embodiment, when a scenario event of a target application is initiated, the target application sends scenario event data to the business processing module, including: when a scenario event of a target application is initiated, the target application sends scenario event data to the business processing module by calling a software program toolkit;

[0193] The scene event data includes the identifier of the target application and the identifier of the scene event.

[0194] In this embodiment, the target application can be a third-party application. The third-party application can integrate a software program toolkit to access the business processing module of the device, so that the third-party application can use the voice control capability provided by the device manufacturer to realize that the device automatically starts sound reception recognition after a scene event of the third-party application occurs, so that the device can instantly respond to the voice commands issued by the user to the third-party application, thereby expanding the scenes and applications supported by the device.

[0195] In an optional embodiment, the target application sends scene event data to the scenario perception module through the business processing module, including: the target application sends the scene event data to the business processing module; the business processing module verifies the scene event data, and if the verification passes, the business processing module sends the scene event data to the scenario perception module.

[0196] In an optional embodiment, the business processing module verifies the scene event data, and if the verification passes, the business processing module sends the scene event data to the context perception module, including:

[0197] The business processing module performs data format verification on the scene event data; if the data format of the scene event data meets the preset format requirements, the business processing module sends the scene event data to the context perception module.

[0198] In this embodiment, the business processing module receives the scene event data of the target application and needs to verify the scene event data. After the verification, the data is transmitted to the situation perception module to ensure the integrity and accuracy of the transmitted data.

[0199] In an optional embodiment, the electronic device further includes a service management module, a sound receiving module and a voice recognition module, and the service processing module controls the voice processing module to be started, including:

[0200] The business processing module sends scene event data to the voice processing module;

[0201] The voice processing module performs voice recognition, including:

[0202] The speech processing module queries the preset instruction words of the scene event based on the scene event data;

[0203] If the preset command word of the scene event is found, the voice processing module performs voice recognition by calling the sound receiving module and the voice recognition module to obtain the voice recognition result.

[0204] Among them, the service management module, the sound receiving module and the voice recognition module are located in the application framework layer of the electronic device.

[0205] In this embodiment, after receiving the scene event data, the voice processing module first queries the validity of the scene event. If the preset instruction word of the scene event is found, the scene event is valid, and the voice processing module starts the sound reception recognition. The voice processing module can call the voice recognition module to obtain the voice recognition result.

[0206] In an optional embodiment, if the speech recognition result hits a preset instruction word of a scene event, the method further includes: the speech processing module sends a control instruction to the target application by calling the service management module; and the target application performs a corresponding service operation based on the control instruction.

[0207] In this embodiment, if the voice command issued by the user is consistent with the preset command word of the scene event, the device immediately responds to the voice command issued by the user and performs the corresponding service operation, thereby improving the efficiency of device voice control. For example, after the alarm rings, the user says "turn off the alarm", and after the voice processing module recognizes the command word, it controls the alarm application to turn off the alarm by calling the service management module.

[0208] In an optional embodiment, the method further includes: after the scene event of the target application ends, the target application sends a first message to the voice processing module by calling the service management module, and the first message is used to notify the end of the scene event; the voice processing module turns off the sound reception recognition based on the first message. Fig.10 Control response of the illustrated embodiment.

[0209] In this embodiment, after the scene event of the target application ends, the service management module can be called to notify the voice processing module to turn off the sound reception to save device power consumption.

[0210] In an optional embodiment, the method also includes: after the situation awareness module learns that the scene event of the target application has ended, the situation awareness module sends a second message to the voice processing module through the business processing module, and the second message is used to notify the end of the scene event; the voice processing module turns off the sound reception recognition based on the second message.

[0211] In this embodiment, the business processing module pre-subscribes to the fence of the end of the scene event of the target application. After the situation awareness module triggers the fence, it notifies the business processing module, and the business processing module notifies the voice processing module to turn off the reception to save device power consumption.

[0212] In an optional embodiment, the method further includes: the voice processing module periodically obtains preset instruction words of the scene events of the target application from the server of the target application; and stores the preset instruction words of the scene events of the target application.

[0213] In this embodiment, the server of the target application provides preset command words for each scene event of the target application. It can be understood that as the target application business develops, more services are provided, the scene events are richer, and the preset command words will be more and more accurate. Therefore, the voice processing module needs to periodically interact with the target application server to obtain the latest preset command words for the scene event to enhance the user's voice control experience.

[0214] It is understandable that, in order to realize the above functions, the above electronic device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0215] The embodiment of the present application can divide the functional modules of the above-mentioned electronic device according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0216] In the case of an integrated unit, Fig.11 A possible structural diagram of the electronic device involved in the above embodiment is shown. The electronic device 1100 includes: a processing unit 1101 , a storage unit 1102 , an audio unit 1103 and a display unit 1104 .

[0217] The processing unit 1101 is used to control and manage the actions of the electronic device 1100. For example, it can be used to parse and recognize the user's voice when the user's voice is received, and / or used for other processes of the technology described herein.

[0218] The storage unit 1102 is used to store program codes and data of the electronic device 1100. For example, it stores preset instruction words, etc. The processing unit 1101 calls the program code stored in the storage unit 1102 to execute each step in the above method embodiment.

[0219] The audio unit 1103 is used to collect the voice uttered by the user and play the voice.

[0220] The display unit 1104 is used to display the interface of the electronic device 1100 .

[0221] Of course, the unit modules in the electronic device 1100 include but are not limited to the processing unit 1101, the storage unit 1102, the audio unit 1103 and the display unit 1104. For example, the electronic device 1100 may also include a power supply unit, a communication unit, etc. The power supply unit is used to supply power to the electronic device 1100. The communication unit is used to support the communication between the electronic device 1100 and other devices.

[0222] Among them, the processing unit 1101 can be a processor or a controller, for example, a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The processor may include an application processor and a baseband processor. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The storage unit 1102 may be a memory. The audio unit 1103 may include a microphone, a speaker, a receiver, etc. The display unit 1104 may be a display screen, etc. The communication unit may be a transceiver, a transceiver circuit or a communication interface, etc.

[0223] For example, the processing unit 1101 is a processor (such as Figure 7 The processor 110 shown in FIG. 10A , the storage unit 1102 may be a memory (eg, Figure 7 The audio unit 1103 may include a microphone (eg, Figure 7 Microphone 170C shown), speaker (such as Figure 7 Speaker 170A shown), receiver (such as Figure 7 The display unit 1104 may be a display screen (eg, Figure 7 The communication unit includes a mobile communication module (such as Figure 7 The mobile communication module 120 shown in FIG. Figure 7 The mobile communication module and the wireless communication module may be collectively referred to as a communication interface.

[0224] The electronic device 1100 provided in the embodiment of the present application can be Figure 7 The electronic device 100 shown in the figure. The processor, memory, microphone, display screen and communication interface can be coupled together, for example, via a bus. The processor calls the program code stored in the memory to execute each step in the above method embodiment.

[0225] An embodiment of the present application provides an electronic device, including: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device performs each function or step in the above method embodiment.

[0226] The present application also provides a chip system (eg, a system on a chip (SoC)), such as Fig.12 As shown, the chip system includes at least one processor 1201 and at least one interface circuit 1202. The processor 1201 and the interface circuit 1202 can be interconnected via lines. For example, the interface circuit 1202 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 1202 can be used to send signals to other devices (such as a processor 1201 or a touch screen of an electronic device). Exemplarily, the interface circuit 1202 can read instructions stored in the memory and send the instructions to the processor 1201. When the instructions are executed by the processor 1201, the electronic device can execute the various steps in the above embodiments. Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0227] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the electronic device executes each function or step in the above method embodiment.

[0228] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the electronic device in the above method embodiment.

[0229] Among them, the electronic device, chip system, computer-readable storage medium or computer program product provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0230] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0231] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0232] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0233] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0234] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0235] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A voice control method, characterized in that: Applied to an electronic device, the electronic device includes a target application, a service processing module, a context awareness module and a voice processing module, and the method includes: In response to the user turning on the voice control switch, the business processing module subscribes to the fence of the scene event of the target application from the context awareness module; When the scene event of the target application is started, the target application sends the scene event data to the context perception module through the business processing module; If the scenario perception module determines that the scenario event has been subscribed based on the scenario event data, the scenario perception module notifies the service processing module, and the service processing module controls to pull up the voice processing module; The speech processing module performs sound reception recognition.

2. The method according to claim 1, characterized in that When the scene event of the target application is started, the target application sends scene event data to the business processing module, including: When the scene event of the target application is started, the target application sends the scene event data to the business processing module by calling the software program toolkit; The scene event data includes an identifier of the target application and an identifier of the scene event.

3. The method according to claim 1 or 2, characterized in that: The target application sends the scene event data to the context perception module through the business processing module, including: The target application sends the scene event data to the business processing module; The business processing module verifies the scene event data, and if the verification passes, the business processing module sends the scene event data to the context perception module.

4. The method according to claim 3, characterized in that The business processing module verifies the scene event data, and if the verification passes, the business processing module sends the scene event data to the context perception module, including: The business processing module performs data format verification on the scene event data; If the data format of the scene event data meets the preset format requirement, the business processing module sends the scene event data to the context perception module.

5. The method according to any one of claims 1 to 4, characterized in that: The electronic device further includes a service management module, a sound receiving module and a voice recognition module, and the service processing module controls the voice processing module to be pulled up, including: The business processing module sends the scene event data to the voice processing module; The speech processing module performs sound reception recognition, including: The voice processing module searches for a preset instruction word of the scene event based on the scene event data; If the preset instruction word of the scene event is found, the voice processing module performs voice recognition by calling the sound receiving module and the voice recognition module to obtain the voice recognition result.

6. The method according to claim 5, characterized in that If the speech recognition result matches the preset instruction word of the scene event, the method further includes: The voice processing module sends a control instruction to the target application by calling the service management module; The target application performs corresponding service operations based on the control instruction.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: After the scene event of the target application ends, the target application sends a first message to the voice processing module through the service management module of the electronic device, where the first message is used to notify the end of the scene event; The voice processing module turns off voice recognition based on the first message.

8. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: After the scenario awareness module learns that the scenario event of the target application ends, the scenario awareness module sends a second message to the voice processing module through the business processing module, where the second message is used to notify the end of the scenario event; The voice processing module turns off voice recognition based on the second message.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: The voice processing module periodically obtains preset instruction words of the scene events of the target application from the server of the target application; The preset instruction words of the scene events of the target application are stored.

10. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images