A processing method and related apparatus

Abstract

The application discloses a processing method. An electronic device includes an application processor and a micro control unit. The micro control unit is deployed with a first application. The micro control unit of the electronic device runs the first application, and sends specified data to the application processor when the micro control unit implements a first function provided by a system service, and implements the first function through a function module provided by the application processor. After the micro control unit implements the first function through the application processor, the micro control unit continues to run the first application. In this way, the electronic device can run the application through the micro control unit, and reduce the power consumption of the electronic device. The electronic device can also implement the first function through the application processor, so that the user can use the first function during the process of running the first application through the micro control unit. It can be understood that the electronic device does not need to deploy the first application on the application processor, but only needs to provide a function framework for implementing the first function, thereby reducing the deployment cost.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a processing method and related apparatus. Background Technology

[0002] With the development of technology, smart wearable devices are becoming increasingly popular. However, because smart wearable devices need to collect and process users' exercise data, physical condition data, and other information for extended periods, and process this data through the application processor, the application processor remains in a wake-up state for a long time, resulting in high power consumption. Therefore, how to reduce the power consumption of smart wearable devices and extend their battery life has become an urgent problem to be solved. Summary of the Invention

[0003] This application provides a processing method and related apparatus that enables the running of an application program through a microcontroller unit of an electronic device, thereby reducing the power consumption of the electronic device; and implements a first function through an application processor, allowing the user to use the first function while the application program is running on the microcontroller unit. It eliminates the need to deploy the first application on the application processor; only a functional framework for implementing the first function needs to be provided, thus reducing deployment costs.

[0004] In a first aspect, this application provides a processing method, wherein the electronic device includes an application processor and a microcontroller unit; the method includes: The application processor controls the display screen to display a first interface, the first interface including one or more application icons, the one or more application icons including the icon of the first application; The application processor, in response to a first input to an icon for the first application, sends a first instruction to the microcontroller unit; After sending the first instruction to the microcontroller, the application processor switches to sleep mode; The microcontroller unit responds to a first instruction and controls the display screen to show the interface of a first application, the interface of the first application including a first control; The microcontroller unit, in response to a second input to the first control, sends a second instruction to the application processor; The application processor responds to the second instruction and controls the microphone to acquire the first audio data; The application processor converts the first audio data into the first text information; The application processor sends the first text information to the microcontroller unit; The microcontroller displays the first text information on the interface of the first application.

[0005] In this way, the electronic device can run the first application through the microcontroller unit and put the application processor into sleep mode to save power. Furthermore, while the microcontroller unit is running the first application, the electronic device can also use the application processor to perform speech-to-text conversion.

[0006] In one possible implementation, the application processor, in response to a first input to an icon for the first application, sends a first instruction to the microcontroller unit, specifically including: In response to the first input, the application processor retrieves the processor identifier of the first application from memory; When the processor identifier of the first application indicates that the microcontroller unit controls the display screen to show the interface of the first application, the application processor sends a first instruction to the microcontroller unit based on the processor identifier of the first application.

[0007] In this way, the application processor can determine whether the application is running through the application processor or the microcontroller unit by using the stored application's processor identifier.

[0008] In one possible implementation, before the application processor responds to the second instruction and controls the microphone to acquire the first audio data, the method further includes: The application processor receives a second instruction and switches from sleep mode to non-sleep mode.

[0009] In this way, the microcontroller can wake up the application processor, allowing the application processor to perform speech-to-text operations.

[0010] In one possible implementation, after the application processor sends the first text information to the microcontroller, the method further includes: the application processor switching to a sleep state.

[0011] In this way, after the application processor sends the converted text information to the microcontroller unit, it can switch back to sleep mode to save power.

[0012] In one possible implementation, the clock frequency of the microcontroller is less than the clock frequency of the application processor, and the operating current of the microcontroller is less than the operating current of the application processor, while the power consumption of the microcontroller is lower than that of the application processor.

[0013] In this way, since the power consumption of the microcontroller is lower than that of the application processor, the power consumption of the electronic device running the primary application is saved.

[0014] Secondly, this application provides an alternative processing method, including: The electronic device includes an application processor and a microcontroller unit, both of which are configured in an accessibility mode; the method includes: The microcontroller controls the display screen to show the interface of the second application, and the first position of the interface of the second application includes second text information or a first control corresponding to the second text information. After receiving the third input for the first position, the microcontroller sends a fourth instruction to the application processor, which includes the second text information. The application processor responds to the fourth instruction and converts the second text information into second audio data; The application processor controls the speaker to broadcast the second audio data.

[0015] In this way, the electronic device can run a second application through the microcontroller unit, saving power. Furthermore, while the microcontroller unit is running the first application, the electronic device can also implement speech-to-text functionality through the application processor.

[0016] In one possible implementation, the microcontroller controls the display screen to show the interface of the second application, specifically including: The application processor controls the display screen to display a first interface, which includes one or more application icons, and the one or more application icons include the icon of a second application. In response to a fourth input for an icon of the second application, the application processor retrieves the processor identifier of the second application from memory, the processor identifier of the second application being used to identify the interface of the second application displayed on the screen controlled by the microcontroller unit; The application processor sends a third instruction to the microcontroller unit based on the processor identifier of the second application. The third instruction includes the identifier of the second application. The microcontroller unit responds to the third command and controls the display screen to show the interface of the second application.

[0017] In this way, the application processor can determine whether the application is running through the application processor or the microcontroller unit by using the stored application's processor identifier.

[0018] In one possible implementation, the application processor switches to sleep mode after sending a third instruction to the microcontroller unit.

[0019] This allows the application processor to switch to sleep mode, further saving power.

[0020] In one possible implementation, after the microcontroller receives a third input for the first position, the method further includes: Based on the touch information of the third input, the microcontroller determines that the third input is used to trigger the application processor to broadcast the second text information. The touch information of the third input includes the first position and the touch time of the third input.

[0021] In this way, the microcontroller unit can determine whether to execute the voice broadcast function based on the touch information input by the third input.

[0022] In one possible implementation, the microcontroller determines, based on the touch information from the third input, that the third input is used to trigger the application processor to read out second text information, specifically including: The microcontroller determines the touch event type of the third input based on the touch time of the third input; When the touch event type of the third input is the first event type, the microcontroller determines that the third input is used to trigger the electronic device to perform the operation of broadcasting the text information indicating the first position; The microcontroller acquires the second text information based on the first position; The microcontroller generates the fourth instruction.

[0023] In this way, the microcontroller unit can determine the corresponding touch event type based on the touch information.

[0024] In one possible implementation, the method also includes: After receiving the fourth instruction, the application processor switches to non-sleep mode.

[0025] In this way, the application processor can switch to non-sleep mode and perform voice broadcast operation after receiving the fourth instruction.

[0026] In one possible implementation, after the microcontroller receives a third input for the first position, the method further includes: The microcontroller sends first information to the application processor, the first information including a first location; Upon receiving the first message, the application processor switches to non-sleep mode; After determining the third input, which is used to trigger the application processor to broadcast the text information indicating the first position, based on the first information, the application processor sends a fifth instruction to the microcontroller unit. The fifth instruction is used to instruct the microcontroller unit to send the text information indicating the first position to the application processor.

[0027] In this way, the application processor can uniformly determine the operation corresponding to the third input and obtain the second text information from the microcontroller unit.

[0028] In one possible implementation, the method further includes the following before the microcontroller sends the fourth instruction to the application processor: After receiving the fifth instruction, the microcontroller unit obtains the second text information based on the first position; The microcontroller generates a fourth instruction based on the second text information.

[0029] In this way, when the microcontroller receives the text information indicating the first position from the application processor, it can obtain the second text information corresponding to the first position and send it to the application processor.

[0030] In one possible implementation, when the first information includes touch information of the third input, the touch information of the third input includes the first position and the touch time of the third input; based on the first information, the application processor determines the text information of the third input used to trigger the application processor to broadcast the first position indication, specifically including: The application processor determines the touch event type of the third input based on the touch time of the third input; When the touch event type of the third input is the first event type, the application processor determines that the third input is used to trigger the electronic device to perform the operation of broadcasting the text information indicating the first location.

[0031] In this way, the application processor can determine the operation triggered by the third input based on the touch information of the third input.

[0032] In one possible implementation, when the first information includes the touch event type of the third input and the first location; the application processor determines, based on the first information, the text information of the third input used to trigger the application processor to broadcast the first location indication, specifically including: When the touch event type of the third input is the first event type, the application processor determines that the third input is used to trigger the electronic device to perform the operation of broadcasting the text information indicating the first position.

[0033] In this way, the application processor can determine the operation triggered by the third input based on the first event type.

[0034] In one possible implementation, before the microcontroller sends the first information to the application processor, the method further includes: The microcontroller determines the touch event type of the third input as the first event type based on the touch time in the touch information of the third input; The microcontroller obtains first information based on the first event type and the first position in the third input touch information.

[0035] In one possible implementation, after the microcontroller sends a fourth instruction to the application processor, the method further includes: the microcontroller controlling the display screen to show a first mark at a first location. In one possible implementation, the clock frequency of the microcontroller is less than the clock frequency of the application processor, and the operating current of the microcontroller is less than the operating current of the application processor, while the power consumption of the microcontroller is lower than that of the application processor.

[0036] Thirdly, this application provides an alternative processing method, including: The first electronic device controls the display screen to display a first interface, the first interface including one or more application icons, the one or more application icons including the icon of the first application; In response to a first input to an icon for a first application, the first electronic device sends a first instruction to the second electronic device; After sending a first instruction to the second electronic device, the first electronic device switches to sleep mode; The first electronic device responds to the second instruction sent by the second electronic device and controls the microphone to collect first audio data; The first electronic device converts the first audio data into first text information; The first electronic device sends the first text information to the second electronic device.

[0037] In one possible implementation, the first electronic device, in response to a first input to an icon for a first application, sends a first instruction to the second electronic device, specifically including: In response to the first input, the first electronic device retrieves the processor identifier of the first application from its memory; When the processor identifier of the first application indicates that the display screen of the first application is controlled by the second electronic device to display the interface of the first application, the first electronic device sends a first instruction to the second electronic device based on the processor identifier of the first application.

[0038] In one possible implementation, before the first electronic device responds to the second instruction and controls the microphone to acquire the first audio data, the method further includes: The first electronic device receives the second instruction and switches from sleep mode to non-sleep mode.

[0039] In one possible implementation, after the first electronic device sends the first text information to the second electronic device, the method further includes: The first electronic device is switched to sleep mode.

[0040] In one possible implementation, the clock frequency of the second electronic device is less than that of the first electronic device, and the operating current of the second electronic device is less than that of the first electronic device, and the power consumption of the second electronic device is lower than that of the first electronic device.

[0041] Fourthly, this application provides an alternative processing method, including: The second electronic device responds to the first instruction sent by the first electronic device and controls the display screen to display the interface of the first application, the interface of the first application including the first control; In response to a second input to the first control, the second electronic device sends a second instruction to the first electronic device; the second instruction is also used to instruct the first electronic device to control the microphone to collect first audio data and convert the first audio data into first text information. After receiving the first text message sent by the first electronic device, the second electronic device displays the first text message on the interface of the first application.

[0042] In one possible implementation, the first instruction specifically refers to: when the processor identifier of the first application indicates that the display screen of the first application is controlled by the second electronic device to display the interface of the first application, the first electronic device sends a first instruction to the second electronic device based on the processor identifier of the first application, wherein the processor identifier of the first application is obtained by the first electronic device from memory in response to the first input.

[0043] In one possible implementation, the second instruction is also used to instruct the first electronic device to switch from a sleep state to a non-sleep state.

[0044] In one possible implementation, the clock frequency of the second electronic device is less than that of the first electronic device, and the operating current of the second electronic device is less than that of the first electronic device, and the power consumption of the second electronic device is lower than that of the first electronic device.

[0045] Fifthly, this application provides an alternative processing method, including: When the first electronic device is in accessibility mode, the first electronic device receives a fourth instruction sent by the second electronic device. The fourth instruction includes second text information. The fourth instruction is sent by the second electronic device to the application processor after receiving a third input for a first location on the interface of the second application.

[0046] The first electronic device responds to the fourth instruction and converts the second text information into second audio data; The first electronic device controls the speaker to broadcast the second audio data.

[0047] In one possible implementation, before the first electronic device receives the fourth instruction sent by the second electronic device, the method further includes: The application processor controls the display screen to display a first interface, which includes one or more application icons, and the one or more application icons include the icon of a second application. In response to a fourth input for an icon of the second application, the application processor retrieves the processor identifier of the second application from memory, the processor identifier of the second application being used to identify the interface of the second application displayed on the screen controlled by the microcontroller unit; The application processor sends a third instruction to the microcontroller unit based on the processor identifier of the second application. The third instruction includes the identifier of the second application and is used to instruct the microcontroller unit to control the display screen to show the interface of the second application.

[0048] In one possible implementation, after the application processor sends a third instruction to the microcontroller, the method further includes switching to a sleep state.

[0049] In one possible implementation, after the application processor receives the fourth instruction, the method also includes switching to a non-sleep state.

[0050] In one possible implementation, the method further includes: before the first electronic device responds to the fourth instruction and converts the second text information into second audio data, the method further includes: The application processor receives first information sent by the microcontroller unit, the first information including a first position; Upon receiving the first message, the application processor switches to non-sleep mode; After determining the third input, which is used to trigger the application processor to broadcast the text information indicating the first position, based on the first information, the application processor sends a fifth instruction to the microcontroller unit. The fifth instruction is used to instruct the microcontroller unit to send the text information indicating the first position to the application processor.

[0051] In one possible implementation, when the first information includes touch information of the third input, the touch information of the third input includes the first position and the touch time of the third input; based on the first information, the application processor determines the text information of the third input used to trigger the application processor to broadcast the first position indication, specifically including: The application processor determines the touch event type of the third input based on the touch time of the third input; When the touch event type of the third input is the first event type, the application processor determines that the third input is used to trigger the electronic device to perform the operation of broadcasting the text information indicating the first location.

[0052] In one possible implementation, when the first information includes the touch event type of the third input and the first location; the application processor determines, based on the first information, the text information of the third input used to trigger the application processor to broadcast the first location indication, specifically including: When the touch event type of the third input is the first event type, the application processor determines that the third input is used to trigger the electronic device to perform the operation of broadcasting the text information indicating the first position.

[0053] In one possible implementation, the clock frequency of the microcontroller is less than the clock frequency of the application processor, and the operating current of the microcontroller is less than the operating current of the application processor, while the power consumption of the microcontroller is lower than that of the application processor.

[0054] Sixthly, this application provides an alternative processing method, including: When the second electronic device is in accessibility mode and controls the display screen to show the interface of the second application, after receiving the third input for the first position of the interface of the second application, it sends a fourth instruction to the first electronic device. The fourth instruction includes second text information. The first position of the interface of the second application includes the second text information or the first control corresponding to the second text information. The fourth instruction is used to instruct the first electronic device to convert the second text information into second audio data and control the application processor to control the speaker to play the second audio data.

[0055] In one possible implementation, the microcontroller controls the display screen to show the interface of the second application, specifically including: The microcontroller responds to a third instruction sent by the application processor to control the display screen to show the interface of the second application. The third instruction is a third instruction sent by the application processor to the microcontroller based on the processor identifier of the second application. The third instruction includes the identifier of the second application. In one possible implementation, after the microcontroller receives a third input for the first position, the method further includes: Based on the touch information of the third input, the microcontroller determines that the third input is used to trigger the application processor to broadcast the second text information. The touch information of the third input includes the first position and the touch time of the third input.

[0056] In one possible implementation, the microcontroller determines, based on the touch information from the third input, that the third input is used to trigger the application processor to read out second text information, specifically including: The microcontroller determines the touch event type of the third input based on the touch time of the third input; When the touch event type of the third input is the first event type, the microcontroller determines that the third input is used to trigger the electronic device to perform the operation of broadcasting the text information indicating the first position; The microcontroller acquires the second text information based on the first position; The microcontroller generates the fourth instruction.

[0057] In one possible implementation, after the microcontroller receives a third input for the first position, the method further includes: The microcontroller sends first information to the application processor, the first information including a first location; The first message is used to instruct the application processor to switch to a non-sleep state; The microcontroller receives a fifth instruction from the application processor. The fifth instruction is generated by the microcontroller based on the first information and after determining the third input to trigger the application processor to broadcast the text information of the first position indication. The fifth instruction is used to instruct the microcontroller to send the text information of the first position indication to the application processor.

[0058] In one possible implementation, the method further includes the following before the microcontroller sends the fourth instruction to the application processor: After receiving the fifth instruction, the microcontroller unit obtains the second text information based on the first position; The microcontroller generates a fourth instruction based on the second text information.

[0059] In one possible implementation, before the microcontroller sends the first information to the application processor, the method further includes: The microcontroller determines the touch event type of the third input as the first event type based on the touch time in the touch information of the third input; The microcontroller obtains first information based on the first event type and the first position in the third input touch information.

[0060] In one possible implementation, after the microcontroller sends a fourth instruction to the application processor, the method further includes: The microcontroller controls the display screen to show the first mark at the first position.

[0061] In one possible implementation, the clock frequency of the microcontroller is less than the clock frequency of the application processor, and the operating current of the microcontroller is less than the operating current of the application processor, while the power consumption of the microcontroller is lower than that of the application processor.

[0062] In a seventh aspect, this application provides a processing apparatus, including an application processor and a microcontroller unit; wherein, An application processor is used to control the display screen to display a first interface, the first interface including one or more application icons, the one or more application icons including the icon of a first application; The application processor is also configured to send a first instruction to the microcontroller unit in response to a first input to an icon for the first application; The application processor is also used to switch to a sleep state after sending the first instruction to the microcontroller unit; A microcontroller unit is configured to, in response to a first instruction, control a display screen to show an interface of a first application, the interface of the first application including a first control. The microcontroller unit is also configured to send a second instruction to the application processor in response to a second input to the first control; The application processor is also used to control the microphone to acquire first audio data in response to a second instruction; The application processor is also used to convert the first audio data into first text information; The application processor is also used to send the first text information to the microcontroller unit; The microcontroller unit is also used to display first text information on the interface of the first application.

[0063] In one possible implementation, the application processor is used to send a first instruction to the microcontroller unit in response to a first input to an icon for the first application. Specifically, it includes: The application processor is used to retrieve the processor identifier of the first application from memory in response to the first input; When the processor identifier of the first application indicates that the microcontroller unit controls the display screen to show the interface of the first application, the application processor is also used to send a first instruction to the microcontroller unit based on the processor identifier of the first application.

[0064] In one possible implementation, before the application processor controls the microphone to acquire first audio data in response to the second instruction, the application processor is further configured to: switch from a sleep state to a non-sleep state in response to the received second instruction.

[0065] In one possible implementation, after the application processor sends the first text information to the microcontroller, the application processor is also used to switch to a sleep state.

[0066] In one possible implementation, the clock frequency of the microcontroller is less than the clock frequency of the application processor, and the operating current of the microcontroller is less than the operating current of the application processor, while the power consumption of the microcontroller is lower than that of the application processor.

[0067] Eighthly, this application provides another processing apparatus, including an application processor and a microcontroller unit; wherein, The microcontroller unit is used to control the display screen to show the interface of the second application, wherein the first position of the interface of the second application includes second text information or a first control corresponding to the second text information. The microcontroller unit is also configured to send a fourth instruction to the application processor after receiving a third input for the first position, the fourth instruction including second text information; An application processor, in response to a fourth instruction, converts the second text information into second audio data; The application processor is also used to control the speaker to broadcast second audio data.

[0068] In one possible implementation, the microcontroller unit is used to control the display screen to show the interface of the second application, specifically including: The application processor is used to control the display screen to display a first interface, the first interface including one or more application icons, and the one or more application icons including the icon of a second application; The application processor is used to retrieve the processor identifier of the second application from memory in response to a fourth input for an icon of the second application. The processor identifier of the second application is used to identify the interface of the second application displayed on the display screen controlled by the microcontroller unit. The application processor is also used to send a third instruction to the microcontroller unit based on the processor identifier of the second application, the third instruction including the identifier of the second application; The microcontroller unit is used to control the display screen to show the interface of the second application in response to a third instruction.

[0069] In one possible implementation, after sending a third instruction to the microcontroller, the application processor is also used to switch to a sleep state.

[0070] In one possible implementation, after the microcontroller receives a third input for the first position, the microcontroller is further configured to determine, based on the touch information of the third input, that the third input is used to trigger the application processor to broadcast second text information, wherein the touch information of the third input includes the first position and the touch time of the third input.

[0071] In one possible implementation, the microcontroller unit is used to determine, based on the touch information from the third input, that the third input is used to trigger the application processor to read out second text information, specifically including: The microcontroller unit is used to determine the touch event type of the third input based on the touch time of the third input; When the touch event type of the third input is the first event type, the microcontroller unit is also used to determine that the third input is used to trigger the electronic device to perform the operation of broadcasting the text information of the first position indication; The microcontroller unit is used to acquire second text information based on a first position; The microcontroller unit is also used to generate fourth instructions.

[0072] In one possible implementation, the application processor, upon receiving the fourth instruction, also switches to a non-sleep state.

[0073] In one possible implementation, after receiving a third input for the first position, the microcontroller is further configured to send first information to the application processor, the first information including the first position; After receiving the first message, the application processor is also used to switch to a non-sleep state; The application processor is also configured to send a fifth instruction to the microcontroller unit after determining, based on the first information, the third input used to trigger the application processor to broadcast the text information of the first position indication. The fifth instruction is used to instruct the microcontroller unit to send the text information of the first position indication to the application processor.

[0074] In one possible implementation, before the microcontroller sends the fourth instruction to the application processor, the microcontroller is also configured to obtain the second text information based on the first position after receiving the fifth instruction. The microcontroller unit is also used to generate a fourth instruction based on the second text information.

[0075] In one possible implementation, when the first information includes touch information of the third input, the touch information of the third input includes the first position and the touch time of the third input; the application processor is used to determine, based on the first information, the text information of the third input used to trigger the application processor to broadcast the first position indication, specifically including: The application processor is used to determine the touch event type of the third input based on the touch time of the third input; When the touch event type of the third input is the first event type, the application processor determines that the third input is used to trigger the electronic device to perform the operation of broadcasting text information indicating the first location.

[0076] In one possible implementation, when the first information includes the touch event type of the third input and the first location; the application processor determines, based on the first information, the text information of the third input used to trigger the application processor to broadcast the first location indication, specifically including: The application processor is used to determine, when the touch event type of the third input is the first event type, that the third input is used to trigger the electronic device to perform the operation of broadcasting text information indicating the first position.

[0077] In one possible implementation, before the microcontroller sends the first information to the application processor, the microcontroller is also used to determine the touch event type of the third input as the first event type based on the touch time in the touch information of the third input; The microcontroller unit is also used to obtain first information based on the first position in the touch information of the first event type and the third input.

[0078] In one possible implementation, after the microcontroller sends a fourth instruction to the application processor, the microcontroller is also used to control the display screen to display a first mark at a first position.

[0079] In one possible implementation, the clock frequency of the microcontroller is less than the clock frequency of the application processor, and the operating current of the microcontroller is less than the operating current of the application processor, while the power consumption of the microcontroller is lower than that of the application processor.

[0080] Ninthly, this application provides an electronic device including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform the processing method in any possible implementation of any of the above aspects.

[0081] In a tenth aspect, embodiments of this application provide a computer storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the processing method in any of the possible implementations of any of the above aspects.

[0082] Eleventhly, embodiments of this application provide a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to execute the processing method in any of the possible implementations of any of the above aspects. Attached Figure Description

[0083] Figure 1 A schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of this application; Figure 2 A schematic diagram of the layered architecture of an application processor and a microcontroller unit of an electronic device 100 provided in an embodiment of this application; Figure 3 A schematic diagram of the layered architecture of an application processor and a microcontroller unit in another electronic device 100 provided in this application embodiment; Figure 4 A flowchart is provided for an embodiment of this application; Figure 5 This is another schematic diagram of a process provided for an embodiment of this application. Detailed Implementation

[0084] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0085] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0086] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with a user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0087] The following describes an electronic device 100 provided in an embodiment of this application.

[0088] Electronic device 100 may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device and / or smart city device. The embodiments of this application do not impose any special restrictions on the specific type of electronic device.

[0089] Please refer to Figure 1 , Figure 1 An exemplary schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of this application is shown.

[0090] like Figure 1As shown, the electronic device 100 may include an application processor (AP) 101, a microcontroller unit (MCU) 102, a power switch 103, a memory 104, etc. These modules can be connected via a bus or other means; this embodiment uses a bus connection as an example.

[0091] Both the application processor 101 and the microcontroller unit 102 can be used to read and execute computer-readable instructions. Furthermore, they can generate operation control signals based on the instruction opcode and timing signals to control instruction fetching and execution.

[0092] In some embodiments, the application processor 101 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0093] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the application processor 101 may include multiple I2C buses. The application processor 101 can couple different devices, such as touch sensors, chargers, flashlights, and cameras, through different I2C bus interfaces. For example, the application processor 101 can couple a touch sensor through the I2C interface, enabling communication between the application processor 101 and the touch sensor via the I2C bus interface, thus realizing the touch function of the electronic device 100.

[0094] The I2S interface can also be used for audio communication. In some embodiments, the application processor 101 may include multiple I2S buses. The application processor 101 can be coupled to the audio module via the I2S bus to realize communication between the application processor 101 and the audio module.

[0095] It should be noted that the microcontroller unit 102 may also include one or more of the aforementioned interfaces. It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0096] In its implementation, the application processor 101 can integrate multiple modules, including a central processing unit (CPU), a graphics processing unit (GPU), a video codec, and a memory subsystem. In some examples, the application processor 101 can be an Arm-Cortex-A core processor with a clock frequency (also known as operating frequency) exceeding 1 GHz, typically comprising four or more processing cores. The application processor 101 has a random access memory capacity of approximately 2 GB or more. The application processor 101 can be used to run operating systems such as Linux, Android, and Windows. The application processor 101 typically draws 100 mAh of current during operation and 4 mAh of current during standby.

[0097] The microcontroller unit 102 may include a central processing unit (CPU), memory, a timer, and one or more interfaces. In some examples, the microcontroller unit 102 may be an Arm-Cortex-M core processor with a clock frequency (also known as operating frequency) of approximately 192 MHz, and the processor core is typically single-core. The random access memory (RAM) of the microcontroller unit 102 has a capacity of approximately 2 MB. The microcontroller unit 102 can support the operation of a lightweight IoT operating system, such as the LiteOS operating system. The operating current of the microcontroller unit 102 is typically 2 mAh, and the standby current is typically 0.1 mAh. For example, the microcontroller unit 102 may be an STL4R9 chip, a Dialog microcontroller, etc.

[0098] The power switch 103 can be used to control the power supply to the electronic device 100.

[0099] Memory 104 is used to store various software programs and / or multiple sets of instructions. In specific implementations, memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some embodiments, the memory in application processor 101 and / or microcontroller unit 102 is a cache memory. This memory can store instructions or data that the processor has just used or that are being used repeatedly. If the processor needs to reuse the instruction or data, it can retrieve it directly from the memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency. In some examples, memory 104 is coupled to application processor 101 and microcontroller unit 102.

[0100] Optionally, the electronic device 100 may also include a display screen. Figure 1 (Not shown in the image), the display screen can be used to display images, videos, controls, text information, etc. The display screen may include a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 205, where N is a positive integer greater than 1.

[0101] Optionally, the electronic device 100 may also include a communication module ( Figure 1 (Not shown in the image), the communication module may include a Bluetooth module, a WLAN module, etc. The electronic device 100 can receive or transmit wireless signals through the communication module. The electronic device 100 can establish a communication connection with other electronic devices through the communication module and interact with them based on this communication connection.

[0102] Optionally, the electronic device 100 may include one or more sensors. For example, the electronic device 100 may include a touch sensor, which may also be referred to as a "touch device". The touch sensor may be disposed on a display screen, and the touch sensor and the display screen constitute a touch screen, which may also be referred to as a "touchscreen". The touch sensor can be used to detect touch operations applied to or near it.

[0103] Optionally, the electronic device 100 may also include an audio module, speaker, microphone, etc. The electronic device 100 can implement audio functions through the audio module, speaker, receiver, microphone, and application processor 101 (or microcontroller unit 102), such as music playback and recording.

[0104] The audio module converts digital audio information into analog audio signals for output, and also converts analog audio input into digital audio signals. The loudspeaker, also called a "horn," converts audio electrical signals into sound signals. The receiver, also called a "handpiece," converts audio electrical signals into sound signals. The microphone, also called a "microphone" or "voice transducer," converts sound signals into electrical signals.

[0105] In some embodiments of this application, the electronic device 100 is a smartwatch, which may further include a watch strap and a watch face. The watch face may include the aforementioned display screen for displaying images, videos, controls, text information, etc. The watch strap can be used to secure the electronic device 100 to the limbs for easy wearing.

[0106] In one possible implementation, the electronic device 100 includes an application processor and a microcontroller unit (MCU). The MCU can process sensor data and send the processed sensor data to the application processor. In this way, the low-power MCU can control the sensor and process the data acquired by the sensor, reducing the workload of the high-power application processor and thus lowering the power consumption of the electronic device 100.

[0107] For example, a microcontroller unit (MCU) can be used to determine the motion posture of an electronic device 100. Specifically, the MCU can determine the motion posture of the electronic device 100 using relevant sensors (e.g., a gyroscope sensor, an accelerometer). For instance, when the electronic device 100 detects a user's motion state, it can detect the angle data of the electronic device 100's rotation using a gyroscope sensor. The MCU can then determine the user's motion posture (e.g., walking, running, sitting, etc.) based on the angle data. The MCU can then send the user's motion posture to an application processor, which can process the user's motion information (e.g., display the user's motion posture on a screen).

[0108] For example, the microcontroller unit can also be used to obtain a user's heart rate value through sensors. For instance, the microcontroller unit can acquire an analog signal value reflecting the pulse heart rate using a photoreflective analog sensor. The microcontroller unit can convert the analog signal value acquired by the photoreflective analog sensor into a digital signal and calculate the heart rate value based on the digital signal.

[0109] It should be noted that the microcontroller unit can control the sensors to acquire the user's body data while the application processor is in sleep mode. This reduces the number of times the application processor is woken up, further reducing power consumption.

[0110] However, since the lower-power microcontroller is only used to process sensor data, the computing power of the microcontroller is not fully utilized, and most operations are still performed by the higher-power application processor, resulting in only a small reduction in the power consumption of electronic device 100.

[0111] To facilitate the description of the following embodiments, the process of the application processor and microcontroller unit running the application will be introduced first.

[0112] Specifically, application processors (e.g., Figure 1 When the application processor 101 shown runs an application, it can retrieve the application's program code from the application processor's memory. The programming language of the program code is a programming language indicated by the application processor (e.g., Java / C++). The application processor can compile the program code into computer instructions using its compiler and execute these computer instructions. Each of these computer instructions belongs to the application processor's instruction set.

[0113] Similarly, microcontrollers (e.g., Figure 1 When the microcontroller unit 102 shown runs an application, it can retrieve the application's program code from its memory. The programming language of the program code is the programming language indicated by the microcontroller unit (e.g., C / C++). The microcontroller unit can compile the program code into computer instructions using its compiler and execute these instructions. Each of these computer instructions belongs to the microcontroller unit's instruction set.

[0114] Understandably, due to the different instruction sets of the microcontroller and the application processor, the programming languages ​​specified for the microcontroller and the application processor may differ in some cases. For the same application, the code installed on the microcontroller and the code installed on the application processor will differ.

[0115] Applications running on an application processor can be called applications deployed on the application processor, or simply applications running on the application processor. Similarly, applications running on a microcontroller unit can be called applications deployed on the microcontroller unit, or simply applications running on the microcontroller unit.

[0116] Understandably, because the instruction set of the application processor is different from that of the microcontroller, the application processor cannot run the microcontroller's application programs, and the microcontroller cannot run the application processor's application programs.

[0117] It should be noted that the memory capacity varies depending on the operating frequency of the application processor and the microcontroller unit. For example, the difference in memory capacity for this operating frequency can be found in [reference needed]. Figure 1 The illustrated embodiments will not be described in detail here. The application processor can be used to run complex computational and network services. For example, the application processor can run applications such as voice communication, online music, local screening for atrial fibrillation and premature beats, and maps. The application processor typically operates at a clock speed of 1 GHz, with a runtime of over 1 second, requiring approximately 5 MB or more of memory. The microcontroller unit can be used to run simple computational services. For example, the microcontroller unit 102 can run applications such as heart rate monitoring, stress monitoring, and daily activity tracking. The microcontroller unit 102 typically operates at a clock speed within 100 MHz, with a runtime of around 100 ms, requiring approximately 100 KB of memory. The programming language of the application code run by the application processor is the programming language indicated by the application processor. The programming language of the application code run by the microcontroller unit is the programming language indicated by the microcontroller unit.

[0118] The runtime is the time required for the application processor or microcontroller unit to switch to sleep mode after performing an operation. Compared to the microcontroller unit, the application processor takes longer to switch to sleep mode and consumes more power.

[0119] In some embodiments, the application is deployed only on the application processor. The electronic device 100 runs the application through the application processor. Specifically, the application processor of the electronic device 100 controls the display screen to show a desktop, which includes the icon of the application deployed only on the application processor. The application processor of the electronic device 100 can run the application and control the display screen to show the interface of the application after receiving input (e.g., a click) for the icon of the application.

[0120] In some embodiments, the application is deployed only on the microcontroller unit. When the application is opened, the electronic device 100 uses the microcontroller unit to run the application. Specifically, the application processor of the electronic device 100 controls the display screen to show a desktop, which includes an icon for the application deployed only on the microcontroller unit. The application processor of the electronic device 100 can notify the microcontroller unit to run the application after receiving input (e.g., a click) for the application's icon. During the running of the application, if the microcontroller unit receives input to close the application, it can notify the application processor. Upon receiving the notification, the application processor can control the display screen to show the desktop. The application processor can also switch to sleep mode after notifying the microcontroller unit to run the application.

[0121] In some embodiments, an application is deployed on both the application processor and the microcontroller unit. That is, both the application processor and the microcontroller unit can run the application. To conserve power, the electronic device 100 preferentially runs the application through the microcontroller unit. Under certain circumstances, the electronic device 100 cannot run the application through the microcontroller unit and must run the application solely through the application processor.

[0122] In some examples, this specific situation occurs when accessibility functionality (also known as voice broadcasting) is detected as enabled. The microcontroller unit cannot perform accessibility functionality, but the application processor can. Specifically, the application processor of electronic device 100 controls the display screen to show a desktop, which includes icons for the application deployed on both the application processor and the microcontroller unit. The application processor of electronic device 100 can determine whether accessibility functionality is enabled upon receiving input (e.g., a click) for the application's icon. If the application processor determines that accessibility functionality is not enabled, it instructs the microcontroller unit to run the application. The microcontroller unit can instruct the application processor to control the display screen to show the desktop upon receiving input to close the application.

[0123] The application processor can directly run the application when it determines that the electronic device 100 has accessibility features enabled. The application processor can also control the display screen to show the desktop after receiving input to close the application while it is running.

[0124] It should be noted that the specific case of enabling accessibility functions is only an example, and this specific case can also be other cases, such as enabling speech-to-text functions, etc. This application embodiment does not limit this.

[0125] In this way, the electronic device 100 can deploy the same application on both the application processor and the microcontroller unit. Under normal circumstances, the application can be run through the microcontroller unit, reducing the device's power consumption. Under specific circumstances, the application can be run through the application processor, ensuring the normal operation of the application.

[0126] In some examples, electronic device 100 includes one or more applications, which may include application 1 and application 2. Application 1 is deployed in the microcontroller unit of electronic device 100, and application 2 is deployed in the application processor of electronic device 100. When electronic device 100 receives input from a user to open application 1, it can run application 1 through the microcontroller unit. When electronic device 100 detects that application 2 has been opened, it can switch from the microcontroller unit to the application processor and run application 2 through the application processor. Application 1 can be an application for performing simple computational tasks. Application 2 can be an application for performing complex computational and network tasks. In this way, electronic device 100 can run part of the application through the microcontroller unit, further reducing the workload of the application processor and lowering the power consumption of electronic device 100.

[0127] For example, application 2 can be a calling application, a smart assistant application, etc. During the operation of application 1, the application processor of electronic device 100 can detect incoming call messages from the calling application (i.e., application 2) or voice commands to wake up the smart assistant application. The application processor of electronic device 100 can then notify the microcontroller to stop running application 1 and run application 2 to process the incoming call message or voice command.

[0128] For example, application 2 can be a desktop application (e.g., a launcher application). When the microcontroller unit of electronic device 100 receives an input to return to the desktop while running application 1, it can instruct the application processor to run application 2. Upon receiving the notification, the application processor of electronic device 100 can run application 2, that is, control the display screen to show the desktop.

[0129] Furthermore, when the electronic device 100 can run application 2 through the application processor, the electronic device 100 receives input from the user to open application 1, and the electronic device 100 can switch from the application processor to the microcontroller unit to run application 1 through the microcontroller unit.

[0130] In one possible implementation, the electronic device 100 includes an application 3, which includes one or more functions, including function 31 and function 32. The application processor of the electronic device 100 has the program for application 3 installed on it, and can run the program to implement functions 31 and 32. The microcontroller unit has the program for application 3 installed on it, and can run the program to implement function 31. It should be noted that the code for the application 3 program in the microcontroller unit is different from the code for the application 3 program in the application processor. Since the microcontroller unit cannot implement function 32 of application 3, when the electronic device 100 is running application 3 to implement function 31, it can run application 3 through the microcontroller unit. When the electronic device 100 is running application 3 to implement function 32, it can switch from the microcontroller unit to the application processor to run application 3. In this way, the electronic device 100 can break down an application that the microcontroller unit cannot run independently into one or more functions, deploying a portion of the application's functions on the microcontroller unit, allowing the microcontroller unit to run the code corresponding to that portion of the function, thus reducing the power consumption of the electronic device 100. Function 31 is for drawing the interface, and function 32 is for background functions (e.g., application data processing functions).

[0131] Once the electronic device 100 switches to the application processor, it can continuously run application 3 using the application processor. If the electronic device 100 closes application 3 while running application 3 through the application processor, or if the electronic device 100 runs an application other than application 3, when the electronic device 100 runs application 3 again, the electronic device 100 will run application 3 through the microcontroller unit to achieve function 31.

[0132] Optionally, when electronic device 100 runs application 3 to achieve function 32 through application processor, electronic device 100 runs application 3 to achieve function 31. Electronic device 100 can switch from application processor to microcontroller unit and run application 3 through microcontroller unit.

[0133] In this way, when running application 3, electronic device 100 can reasonably switch the chip running application 3, which saves the power consumption of electronic device 100 and does not affect the use of application 3. However, since the application processor and the application deployed on the microcontroller have different programming languages, repeatedly deploying the code of application 3 on the application processor and the microcontroller not only occupies more storage space, but also increases the difficulty of application deployment, making it inconvenient for electronic device 100 to install more applications.

[0134] This application provides a processing method. An electronic device 100 includes an application processor and a microcontroller unit (MCU). The MCU deploys a first application. The MCU of the electronic device 100 runs the first application. When the MCU implements a first function provided by a system service, it sends specified data to the application processor, and the first function is implemented through the functional modules provided by the application processor. After the application processor implements the first function, the MCU continues to run the first application. In this way, the electronic device 100 can run the application through the MCU, reducing the power consumption of the electronic device 100. The electronic device 100 can also implement the first function through the application processor, allowing the user to use the first function while the MCU is running the first application. It is understood that the electronic device 100 does not need to deploy the first application on the application processor; it only needs to provide a functional framework for implementing the first function, reducing deployment costs.

[0135] When the application processor is in sleep mode (also known as standby mode), its current consumption is low, and its power consumption is low. When the application processor is not in sleep mode, its current consumption is high, and its power consumption is high. It is understandable that the power consumption of an application processor in sleep mode is higher than that of an application processor in non-sleep mode. When the application processor of electronic device 100 is running an application, it is in non-sleep mode, resulting in higher power consumption. When the microcontroller unit of electronic device 100 is running an application, its application processor is in sleep mode, resulting in lower power consumption.

[0136] Next, combine Figure 1 The hardware structure of the electronic device 100 shown exemplifies the interaction flow between the application processor and the microcontroller unit.

[0137] For example, such as Figure 2 As shown, a layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the operating system includes, but is not limited to, an application layer, a framework and service layer (also known as an application framework layer), and a kernel layer. The application layer may include a series of application packages. The framework and service layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer; the framework and service layer includes some predefined functions. The kernel layer is the layer between hardware and software.

[0138] Here, the framework and services deployed on the application processor 101 may include, but are not limited to, the functional module 21. The functional module 21 may be used to implement a first function, such as a speech-to-text (STT) function, a text-to-speech (TTS) function, etc.

[0139] Application processor 101 may also be equipped with a communication module 22, through which application processor 101 and microcontroller unit 102 can transmit data. In some examples, communication module 22 can provide a dual-core communication implementation to enable data transmission between the two processors. In some examples, functional module 21 can obtain data from microcontroller unit 102 through communication module 22 and process the data to achieve a first function. Optionally, functional module 21 can return the processing result (e.g., processed data) to microcontroller unit 102 through communication module 22.

[0140] Optionally, the application processor 101 may also be equipped with a kernel driver module 23, through which the application processor 101 can call corresponding hardware. The kernel driver module 23 may include drivers for the electronic device 100, such as, but not limited to, sensor drivers, speaker drivers, microphone drivers, touchscreen drivers, etc., on the electronic device 100. In some examples, the functional module 21 can call the corresponding hardware through the kernel driver module 23 to implement a first function. For example, the functional module 21 can control the speaker to play the voice data obtained by the functional module 21 based on text through the speaker driver module of the kernel driver module 23, thus implementing a text-to-speech function. As another example, the functional module 21 can control the microphone to collect nearby sound signals through the microphone driver module of the kernel driver module 23, and the functional module 21 can convert the sound signals collected by the microphone into text information, thus implementing a speech-to-text function.

[0141] The application deployed on the microcontroller unit 102 may include a first application. The microcontroller unit 102 can run the program of the first application to implement some or all of its functions. For example, the first application may be a music playback application, a communication application, a sports and health application, etc.

[0142] The framework and services deployed on the microcontroller unit 102 include a functional module 25, which can be used in conjunction with the functional module 21 of the application processor A to implement a first function. In some examples, the functional module 25 can receive data from the first application and transmit that data to the application processor 101 via the communication module 26. Optionally, the functional module 25 can obtain the processing result sent by the application processor 101 through the communication module 26.

[0143] The microcontroller unit 102 may also be equipped with a communication module 26, through which the microcontroller unit 102 and the application processor 101 can transmit data. In some examples, the communication module 26 may provide an offload dual-core communication implementation.

[0144] Optionally, the microcontroller unit 102 may also be equipped with a kernel driver module 27, through which the application processor 101 can call the corresponding hardware. The kernel driver module 27 may include drivers for the electronic device 100, such as, but not limited to, sensor drivers, speaker drivers, microphone drivers, touchscreen drivers, etc., on the electronic device 100. In some examples, the functional module 25 can receive the processing results sent by the application processor 101 through the communication module 26, and then call the corresponding hardware through the kernel driver module 27 to implement the first function. For example, the processing result may be text obtained from speech conversion; the functional module 25 can control the display screen to display the text through the display driver in the kernel driver module 23, thus realizing the speech-to-text function.

[0145] In some embodiments, the first function can be an accessibility function (also known as a screen reading function). When the accessibility function is enabled, the electronic device 100 can read aloud the text information selected by the user, allowing the user to determine the content displayed on the screen without looking at it. The selected text information can be text located at the user's touch point, or it can be text information corresponding to an icon located at the user's touch point (e.g., the name of an application corresponding to an application icon on the desktop). It is understood that since the accessibility function can provide screen reading functionality, it can be understood that the accessibility function includes text-to-speech functionality.

[0146] The electronic device 100 may provide a control for enabling accessibility functionality, which may be provided by a settings application of the electronic device 100. The electronic device 100 may receive input (e.g., a click) to the control for enabling accessibility functionality, and in response to this input, enable the accessibility functionality. When the electronic device 100 enables accessibility functionality, both the application processor 101 and the microcontroller unit 102 are set to accessibility mode.

[0147] Specifically, the touch commands for touch operations when accessibility is enabled differ from those when accessibility is disabled. For example, when accessibility is enabled, a single tap triggers the corresponding text for the targeted icon. When accessibility is disabled, a single tap triggers the corresponding icon's functionality. Similarly, when accessibility is enabled, a double tap triggers the corresponding icon's functionality; when accessibility is disabled, a double tap may trigger the corresponding icon's functionality, or it may not trigger any operation on the electronic device 100, or it may trigger both the double tap and the corresponding icon's functionality again.

[0148] In some examples, when accessibility functionality is enabled, both application processor 101 and microcontroller 102 are set to accessibility mode. To facilitate accessibility functionality for both applications deployed on microcontroller 102 and application processor 101, microcontroller 102 can send user operation information to application processor 101 upon detection. Application processor 101 can then process this operation information using relevant accessibility modules to obtain the corresponding touch command. Application processor 101 can execute the touch command corresponding to this operation information, retrieve text information from microcontroller 102, convert the text information into voice data, and then control the speaker to play the voice data, thus achieving accessibility functionality. In this way, by determining the command corresponding to the user operation through application processor 101, it is not necessary to establish a mapping between user operation information and commands executed by electronic device 100 after accessibility functionality is enabled on both application processor 101 and microcontroller 102. This facilitates unified scheduling of accessibility functionality provided by application processor 101 by electronic device 100 and saves storage space on microcontroller 102.

[0149] In other examples, when the microcontroller unit 102 detects user operation information, it sends the corresponding text information to the application processor 101. The application processor 101 can then process this text information using accessibility-related modules to obtain voice data and control the speaker to play the voice data, thus achieving accessibility functionality. Because the power consumption of the microcontroller unit 102 is lower than that of the application processor 101, using the microcontroller unit 102 to determine the instruction corresponding to the user operation can further reduce the operations performed by the application processor 101, thereby reducing the power consumption of the electronic device 100.

[0150] In some embodiments, functional module 21 may include, for example, an accessibility module 301 and a speech-to-text module 302. Functional module 25 may include, for example, an accessibility module 303 and a speech-to-text module 304. Accessibility functionality is implemented with one module each in the microcontroller and the application processor; the module deployed in the application processor is accessibility module 301, and the module deployed in the microcontroller is accessibility module 303. When the electronic device 100 runs an application using the microcontroller, it can implement accessibility functionality through accessibility modules 303 and 301. When the electronic device 100 runs an application using the application processor, it can implement accessibility functionality solely through accessibility framework A. That is, when the electronic device 100 can run an application using the microcontroller, it can play the text displayed on the screen of the electronic device 100 in speech form through accessibility modules 303 and 301.

[0151] The speech-to-text function has one module deployed in the microcontroller unit and one in the application processor. The module deployed in the application processor is speech-to-text module 302, and the module deployed in the microcontroller unit is speech-to-text module 304. When the electronic device 100 is running an application using the microcontroller unit, it can implement the speech-to-text function through speech-to-text modules 304 and 302. In other words, when the electronic device 100 is running an application using the microcontroller unit, it can convert the voice data collected by the electronic device 100 into text using speech-to-text modules 304 and 302, and display the converted text on the display screen of the electronic device 100.

[0152] Thus, when the electronic device 100 runs an application using a microcontroller, it can utilize the accessibility modules 301 and 303 to implement accessibility functionality using the application processor, and broadcast the content of the first application. The electronic device 100 also utilizes the speech-to-text modules 302 and 304 to implement speech-to-text functionality using the application processor, obtaining and displaying the text information input by the user's voice.

[0153] For example, such as Figure 3 As shown, the framework and services deployed on the application processor 101 may include, but are not limited to, an accessibility module 301, a speech-to-text module 302, a text-to-speech (TTS) engine, and a speech-to-text (STT) engine. The application processor 101 may also deploy a communication module 22 and a kernel driver module 23. The kernel driver module 23 may include a speaker driver module, a microphone driver module, etc. For detailed descriptions of the communication module 22 and the kernel driver module 23, please refer to... Figure 2 The illustrated embodiments will not be described in detail here. It is understood that... Figure 2 The functional module 21 shown can be either an accessibility module 301 or a speech-to-text module 302. The accessibility module 301 can provide an interface for the microcontroller unit 102 to perform text-to-speech functions. The speech-to-text module 302 can also provide an interface for the microcontroller unit 102 to perform speech-to-text functions.

[0154] The microcontroller unit 102 can deploy one or more applications, which may include a first application. The framework and services deployed on the microcontroller unit 102 may include, but are not limited to, an accessibility module 303, a speech-to-text module 304, etc. The microcontroller unit 102 may also deploy a communication module 26 and a kernel driver module 27. The kernel driver module 27 may include a touchscreen driver module, which can be used to detect user input to the touchscreen and control the touchscreen to display the interface of the first application. For detailed descriptions of the communication module 26 and the kernel driver module 27, please refer to [link to relevant documentation]. Figure 2 The illustrated embodiments will not be described in detail here. It is understood that... Figure 2 The functional module 25 shown can be either the accessibility module 303 or the speech-to-text module 304. The specific interaction flow between the various modules of the application processor 101 and the microcontroller unit 102 can be found in [reference needed]. Figure 4 , Figure 5 The embodiments shown are not described in detail here.

[0155] In some embodiments, the electronic device 100 stores a processor identifier for an application. This processor identifier can be used to indicate whether the electronic device 100 runs the application via an application processor or a microcontroller unit. That is, when the electronic device 100 receives input from a user indicating that an application has been opened, it can determine whether the application is deployed on the microcontroller unit or the application processor based on the application's processor identifier, and then run the application via the chip indicated by the processor identifier. In this way, the electronic device 100 can deploy applications on the microcontroller unit and assign corresponding processor identifiers to these applications. When a user opens an application, the electronic device 100 can quickly determine whether to run the application using the microcontroller unit, saving device power consumption. For example, as shown in Table 1, Table 1 illustrates examples of applications and their corresponding processor identifiers.

[0156] Table 1

[0157] As shown in Table 1 above, this table displays some applications and their corresponding processor identifiers. The processor identifiers for the exercise application and the information application are microcontroller units. The processor identifiers for the desktop application and the map application are application processors. That is, when the electronic device 100 displays the desktop, the application processor runs the desktop application and controls the display screen to show the desktop. When the application processor of the electronic device 100 receives input to open an application, it can determine the processor identifier of that application and decide whether to notify the microcontroller unit to run the application. For example, when the application processor of the electronic device 100 is controlling the display screen to show the desktop and receives input to open the information application, the application processor, based on the processor identifier of the information application, determines that the microcontroller unit should run the information application. The application processor can send a notification message to the microcontroller unit, which can carry the identifier of the information application (e.g., the application name). After receiving the notification message from the application processor, the microcontroller unit can run the information application, for example, control the display screen to show the interface of the information application.

[0158] The following section describes the processing method provided in the embodiments of this application, using application scenarios as examples.

[0159] In some application scenarios, electronic device 100 is used as a watch or bracelet, and electronic device 100 includes... Figure 3 The application processor 101 and microcontroller unit 102 are shown, and the first application is an exercise application, which is used as an example to introduce the processing method provided by the embodiments of this application. The exercise application is deployed on the microcontroller unit, which does not support the independent implementation of accessibility functionality; this accessibility functionality is jointly implemented by the application processor and the microcontroller unit. The accessibility functionality is a function provided by the operating system (also known as a system service).

[0160] For example, such as Figure 4 As shown, after the accessibility function of the electronic device 100 is activated, the electronic device 100 can implement the accessibility function through the application processor 101 while running the first application through the microcontroller unit 102. This process can be divided into the following 5 steps: Step 401: The application processor 101 of the electronic device 100 controls the display screen to show the desktop 501, which includes one or more application icons, including an exercise application icon 502. The electronic device 100 enables accessibility functionality. When accessibility functionality is enabled, the electronic device 100 can read aloud the text information selected by the user, allowing the user to determine the content displayed on the screen without looking at it.

[0161] In some examples, the application processor 101 of electronic device 100 deploys a launcher application (also known as a desktop application). The launcher application can be used to manage and display the desktop and application icons on it. The value of the processor identifier of the launcher application can instruct electronic device 100 to run the launcher application using application processor 101. When electronic device 100 receives input from a user to open an exercise application, application processor 101 can determine, based on the exercise application's processor identifier, that the application is to be run by microcontroller unit 102, and application processor 101 of electronic device 100 can instruct microcontroller unit 102 to run the exercise application.

[0162] Optionally, the electronic device 100 can also display accessibility prompts. These prompts can inform the user that accessibility functionality is enabled on the electronic device 100, and can be triggered by user input to be read aloud. The accessibility prompts can include, but are not limited to, text-based, voice-based, and animated prompts. Further optionally, the accessibility prompts can also instruct the user on how to disable accessibility mode. For example, the prompt could be a text message such as, "Currently in accessibility mode. Double-click the power button to exit." The electronic device 100 can also read aloud the accessibility prompts.

[0163] Step 402: After receiving the input (e.g., double-click) for the exercise application icon 502, the electronic device 100, in response to the input, displays as shown below. Figure 4 The exercise application interface shown is 511.

[0164] The electronic device 100 stores the processor identifier of the application. The application processor 101 of the electronic device 100 controls the display screen to show the desktop 501 and receives input for the exercise application icon 502. Responding to this input, the application processor 101 determines that when accessibility functionality is enabled, the input for the exercise application icon 502 is the input that triggers the electronic device 100 to display the interface of the exercise application icon 502. Then, based on the processor identifier of the exercise application, the application processor 101 can notify the microcontroller unit 102 to run the exercise application. Upon receiving the notification, the microcontroller unit 102 controls the display screen to show the exercise application interface 511. After notifying the microcontroller unit 102 to run the exercise application, the application processor 101 can enter a sleep state to reduce device power consumption.

[0165] The exercise application interface 511 is the interface of the exercise application, which includes one or more icons displayed in the exercise application, including a distance icon 512. Optionally, accessibility prompts may also be displayed at the top of the exercise application interface 511.

[0166] In some embodiments, when accessibility functionality is enabled, application processor 101 is responsible for implementing it. Microcontroller unit 102 can send user touch operation information to application processor 101, which then determines the corresponding instruction. Application processor 101 can execute this instruction to implement accessibility functionality. Specifically, as shown... Figure 4 Steps 403 and 404 are shown in the diagram.

[0167] Step 403: Electronic device 100 can receive user input of text information about distance icon 512 (e.g., a click on distance icon 512). The microcontroller unit 102 of electronic device 100 instructs application processor 101 to convert the text information of distance icon 512 in the exercise application into audio data. Specifically, as... Figure 4 As shown in step 403, step 403 includes multiple sub-steps, which are: Figure 4 Sub-steps 1 to 15 are marked with the middle arrow: Sub-step 1: The touch screen driver module 42 of the microcontroller unit 102 can detect the user's touch operation through the touch sensor of the touch screen.

[0168] After detecting a user's touch operation, the touch screen driver module 42 can process the touch operation information to obtain the touch event type. This touch operation information may include, but is not limited to, the position information of the screen touch point and the timestamp of the touch operation.

[0169] The touch screen driver module 42 of the microcontroller unit 102 can obtain the touch event type based on the timestamp of the touch operation. The touch event type can represent the type of user operation, such as single click, double click, long press, swipe, etc.

[0170] The touch screen driver module 42 can send the touch event type and screen touch point location information to the communication module 26 of the microcontroller unit 102.

[0171] Sub-step 2: After receiving the touch event type and screen touch point location information sent by the touch screen driver module 42, the communication module 26 can send the touch event type and screen touch point location information to the communication module 22 of the application processor 101.

[0172] The communication module 26 can be used to send data from the microcontroller unit 102 to the communication module 22, and can also be used to receive data from the application processor 101 sent by the communication module 22. The electronic device 100 can realize the transmission of data between the microcontroller unit 102 and the application processor 101 through the communication modules 22 and 26.

[0173] It should be noted that after the communication module 22 of the application processor 101 receives the information sent by the communication module 26, the application processor 101 switches from the sleep state to the non-sleep state.

[0174] Sub-step 3: The communication module 22 can send the touch event type and screen touch point location to the accessibility module 301. The accessibility module 301 can determine the corresponding touch command based on the touch event type. The touch command can be used to instruct the microcontroller unit 102 to perform the corresponding operation.

[0175] Here, the touch screen driver module 42 determines the touch event type as a single click based on the touch operation information. The accessibility module 301 identifies the touch instruction corresponding to the single click as an instruction to announce the text at the touch point location on the screen (i.e., an instruction to announce the text information at the distance from icon 512). This touch instruction can include the position information of the screen touch point, which can be used to determine the text to be announced. Here, this touch instruction can be called an accessibility instruction.

[0176] Understandably, when the touch event type obtained by the touch screen driver module 42 based on the touch operation information is a double-click, the touch screen driver module 42 can send the touch event type and the screen touch point position to the accessibility module 301. The accessibility module 301 recognizes that the touch command corresponding to the double-click is an instruction to trigger the function of the icon at the screen touch point position. This touch command can include the screen touch point position information, which can be used to determine the selected icon. After receiving the touch command, the microcontroller unit 102 can determine that the icon at the screen touch point position is the distance icon 512 based on the screen touch point position information. The microcontroller unit 102 (in the exercise application) can execute the function of the distance icon 512. For example, the microcontroller unit 102 can control the display screen to show the distance the user has moved forward.

[0177] When the touch event type obtained by the touch screen driver module 42 based on the touch operation information is knuckle tap, the touch screen driver module 42 can send the touch event type and screen touch point position to the accessibility module 301. The accessibility module 301 recognizes that there is no corresponding touch command for the knuckle tap. The application processor 101 will not execute the operation of sending the touch command to the microcontroller unit 102.

[0178] It should be noted that touch event types such as single click, double click, and knuckle tap are merely examples and should not be construed as specific limitations on the touch commands corresponding to these touch event types. Other touch event types may correspond to different touch commands, or may not have corresponding touch commands; this application embodiment does not impose any limitations on this. It should also be noted that regardless of whether the touch event type is a single click, double click, or other operation, the touchscreen driver module 42 can send the touch event type and the position information of the screen touch point to the accessibility module 301, which then determines the touch command corresponding to the touch event type. Once the accessibility module 301 determines the touch command, it can send the touch command to the microcontroller unit 102, which can then execute the operation indicated by the touch command.

[0179] Optionally, the microcontroller unit 102 obtaining touch operation information for the distance icon 512 via the touch screen driver module 42 is merely an example. In this embodiment, other modules can also be used to recognize and announce the text information of the distance icon 512. For example, the electronic device 100 can receive input from a user on a physical button of the electronic device 100, switch the marker to another icon, and announce the text information of the currently marked icon. Here, the input can be a press input on a physical button of the electronic device 100 (e.g., the power button), triggering the electronic device 100 to announce the text information of the distance icon 512; this application does not limit this.

[0180] In other examples, the application processor 101 of the electronic device 100 includes a touch screen driver module 41, which can process the timestamp of the touch operation to obtain the touch event type. It should be noted that in some examples, the following steps can replace sub-steps 1 to 3 described above.

[0181] Specifically, the touchscreen driver module 42 of the microcontroller unit 102 can detect the user's touch operation through the touch sensor of the touchscreen. This touch operation is the user's action of reading the text information of the distance icon 512. After detecting the user's touch operation, the touchscreen driver module 42 can send the acquired touch operation information to the communication module 26 of the microcontroller unit 102. It can be understood that when the accessibility function is enabled, both the application processor 101 and the microcontroller unit 102 are set to accessibility mode. The touchscreen driver module 42 of the microcontroller unit 102 can send the touch operation information to the application processor 101.

[0182] After receiving the touch operation information sent by the touch screen driver module 42, the communication module 26 can send the touch operation information to the communication module 22 of the application processor 101.

[0183] After receiving the touch operation information sent by the communication module 26, the communication module 22 can write the touch operation information into the touch screen driver module 41. The touch screen driver module 41 can determine the touch event type based on the touch operation information. The touch screen driver module 41 can also obtain the touch event type based on the timestamp of the touch operation. The touch screen driver module 41 can send the touch event type and screen touch point position to the accessibility module 301, and the accessibility module 301 can determine the corresponding touch command based on the touch event type.

[0184] Here, the accessibility module 301 determines that the touch instruction is an instruction to read the text at the screen touch point location. The touch instruction includes the screen touch point location information, which can be used to determine the text to be read. Here, this touch instruction can be called an accessibility instruction. A detailed description of how the accessibility module 301 determines the touch instruction can be found in the description of sub-step 3 above, and will not be repeated here.

[0185] Sub-step 4: After the accessibility module 301 determines the accessibility instruction, it can send the accessibility instruction to the communication module 22.

[0186] Sub-step 5: Communication module 22 sends the accessibility command to communication module 26.

[0187] Sub-step 6: After receiving the accessibility instruction, the communication module 26 can send the accessibility instruction to the accessibility module 303.

[0188] Sub-step 7: The accessibility module 303 can, based on accessibility instructions, instruct the image module to draw the interface displayed by the electronic device 100 in response to the accessibility instructions. Specifically, the accessibility module 303 can, based on accessibility instructions, instruct the image module to draw a marker symbol (i.e., marker 513) at the location indicated by the position information of the screen touch point on the screen frame displayed on the display screen (i.e., interface 511) (i.e., distance from icon 512).

[0189] Sub-step 8: The image module can send a request to the exercise application to obtain the interface data provided by the exercise application. After receiving the request, the exercise application can send the interface data to the image module.

[0190] The request information sent by the image module to the exercise application may include the touch event type and the location information of the screen touch point. The exercise application can determine the interface data required for the next display based on the touch event type and the location information of the screen touch point, and send the interface data to the image module.

[0191] Sub-step 9: The image module can receive interface data sent by the exercise application and draw the exercise application interface 511 based on this interface data. Simultaneously, the image module can also draw at the distance icon 512 indicated by the screen touch point's position information, such as... Figure 4 The marker 513 is shown. This marker 513 can be used to indicate that the distance icon 512 is marked.

[0192] Sub-step 10: The accessibility module 303 can also obtain the text information of the distance icon 512, which is indicated by the position information of the screen touch point, from the image module: "Distance".

[0193] Optionally, the accessibility instructions may not include the location information of the screen touch points, and the microcontroller unit 102 can directly obtain the location information of the screen touch points from the touch screen driver module 42.

[0194] Sub-step 11: The accessibility module 303 can send text information to the communication module 26.

[0195] Sub-step 12: Communication module 26 can send text information to communication module 22.

[0196] Sub-step 13: Communication module 22 can send text information to accessibility module 301.

[0197] Sub-step 14: Accessibility module 301 can send text information to the TTS engine.

[0198] Sub-step 15: The TTS engine can convert text information into audio data. For example, the TTS engine may include a text-to-speech algorithm, which can be used to convert text into speech. The TTS engine can then send the audio data to the speaker driver module, which can control the speaker to play the audio data.

[0199] In some examples, accessibility module 301 can invoke the TTS engine through an accessibility service (e.g., the talkback service) to convert text information into audio data. The accessibility service then plays the audio data through the speaker driver module.

[0200] Step 404: The electronic device 100 can display the exercise application interface 511 and marker 513, and the electronic device 100 can play the audio data. Here, the content played by the electronic device 100 can be: "distance".

[0201] In other embodiments, when the microcontroller unit 102 detects that the touch command corresponding to the user's touch operation is a command to broadcast text information at the screen touch point, it sends the text information at the screen touch point to the application processor 101. The application processor 101 can use the relevant modules of the accessibility function to process the text information, obtain voice data, and control the speaker to play the voice data to realize the accessibility function. It should be noted that the following steps can replace the above. Figure 4 Steps 403 and 404 are shown.

[0202] Specifically, the touch screen driver module 42 can detect the user's touch operation through the touch sensor of the touch screen.

[0203] The touch screen driver module 42 can obtain the touch event type based on the timestamp of the touch operation information.

[0204] The touch screen driver module 42 can send the touch event type and screen touch point location information to the accessibility module 303 of the microcontroller unit 102.

[0205] The accessibility module 303 can determine the corresponding touch command based on the touch event type and execute the operation indicated by the touch command.

[0206] Here, the touch screen driver module 42 determines the touch event type as a single click based on the touch operation information. The accessibility module 303 identifies the touch instruction corresponding to the single click as an instruction to announce the text at the touch point location on the screen (i.e., an instruction to announce the text information at the distance from icon 512). The touch instruction includes the position information of the screen touch point, which can be used to determine the text to be announced. Here, this touch instruction can be called an accessibility instruction.

[0207] Accessibility module 303 can, based on accessibility instructions, instruct the image module to draw the interface displayed by electronic device 100 in response to accessibility instructions. Specifically, accessibility module 303 can, based on accessibility instructions, instruct the image module to draw a marker symbol (i.e., mark 513) at the location indicated by the position information of the screen touch point on the screen frame (i.e., interface 511) displayed on the screen (i.e., distance from icon 512).

[0208] The image module can send a request to the exercise application to obtain the interface data provided by the exercise application. After receiving the request, the exercise application can send the interface data to the image module.

[0209] The image module can receive interface data sent by the exercise application and draw the exercise application interface 511 based on this interface data. Simultaneously, the image module can also draw at the distance icon 512 indicated by the screen touch point's position information, such as... Figure 4 The marker 513 is shown. This marker 513 can be used to indicate that the distance icon 512 is marked.

[0210] The image module can control the display screen of the electronic device 100 to display the exercise application interface 511 and markers 513.

[0211] Accessibility module 303 can also obtain the text information of the distance icon 512, which indicates the position information of the screen touch point, from the image module: "Distance".

[0212] The accessibility module 303 can send text information to the communication module 26.

[0213] Communication module 26 can send text information to communication module 22.

[0214] The communication module 22 can send text information to the accessibility module 301.

[0215] The accessibility module 301 can send text information to the TTS engine.

[0216] A TTS engine can convert text information into audio data.

[0217] The TTS engine can send audio data to the speaker driver module, which can then control the speakers to play that audio data. Here, the content played by the electronic device 100 could be: "distance".

[0218] In this way, the electronic device 100 can draw the interface of the exercise application through the microcontroller unit 102 and obtain the audio data corresponding to the selected text information through the application processor 101.

[0219] Optionally, when the electronic device 100 displays the text content of the distance icon 512, it may also display operation prompts. These operation prompts can be used to guide the user on the operations that the distance icon 512 can perform. For example, the operation prompt could be: "Double-tap with one finger to perform the operation; double-tap and hold to bring up more options."

[0220] It should be noted that the first application is a training application, which is only an example. The first application can also be other applications deployed on the microcontroller unit, such as calendar applications, calculator applications, etc.

[0221] Understandably, when the touch event type obtained by the touch screen driver module 42 based on the touch operation information is a double-click, the accessibility module 303 recognizes that the touch command corresponding to the double-click is an instruction to trigger the function of the icon at the screen touch point location. This touch command may include the screen touch point location information, which can be used to determine the selected icon. The exercise application can determine the icon at the screen touch point location as the distance icon 512 based on the screen touch point location information. The microcontroller unit 102 (of the exercise application) can execute the function of the distance icon 512. For example, the microcontroller unit 102 can control the display screen to show the distance the user has traveled. In this scenario, the microcontroller unit 102 will not wake up the application processor 101.

[0222] It is also understandable that when the touch event type obtained by the touch screen driver module 42 based on the touch operation information is knuckle tap, the touch screen driver module 42 can send the touch event type and screen touch point position to the accessibility module 303. The accessibility module 303 recognizes that there is no corresponding touch command for the knuckle tap, and the micro control unit 102 continues to control the display screen to display the currently displayed interface.

[0223] It should be noted that touch event types such as single click, double click, and knuckle tap are merely examples and should not be construed as specific limitations on the touch commands corresponding to these touch event types. Other touch event types may correspond to different touch commands, or may not have corresponding touch commands; this application embodiment does not limit this. It should also be noted that regardless of whether the touch event type is a single click, double click, or other operation, the touchscreen driver module 42 can send the touch event type and the position information of the screen touch point to the accessibility module 303, which then determines the touch command corresponding to the touch event type. The microcontroller unit 102 can execute the operation indicated by the touch command.

[0224] In other application scenarios, electronic device 100 is a watch or bracelet, and electronic device 100 includes... Figure 3The application processor 101 and microcontroller unit 102 are shown, and the processing method provided in this application embodiment is introduced using an information application as an example. The information application is deployed on the microcontroller unit, which does not support independent implementation of the speech-to-text function; this function is jointly implemented by the application processor and the microcontroller unit.

[0225] For example, the application processor 101 of the electronic device 100 controls the display screen to display, such as Figure 4 As shown on desktop 501, when the application processor 101 of electronic device 100 receives input from the user indicating that a messaging application has been opened, it can determine, based on the processor identifier of the messaging application, that the microcontroller unit 102 is running the messaging application, and then notify the microcontroller unit 102 to run the application. The application processor 101 can also switch from a non-sleep state to a sleep state after notifying the microcontroller unit 102 to run the messaging application. During the process of running the messaging application through the microcontroller unit 102, electronic device 100 can implement speech-to-text functionality through the application processor 101, such as... Figure 5 As shown, the process can be divided into four steps: Step 501: The microcontroller unit 102 of the electronic device 100 controls the display screen to show the information application interface 601. The information application interface 601 may include one or more information options. The information options can be used to trigger the electronic device 100 to display an information details interface for viewing and replying to information. The one or more information options include information option 602. Information option 602 may include, but is not limited to, a contact identifier 603. The contact identifier 603 can be used to indicate the contact person corresponding to information option 602. Here, the contact identifier 603 is "Alice". Optionally, information option 602 may also include an information summary 605, which may include the content of the information sent by the contact person. Optionally, information option 602 may also include a message quantity identifier 604, which can be used to indicate the number of messages from the contact person that the user has not viewed.

[0226] Step 502: After receiving input for information option 602, electronic device 100 responds to the input by displaying, as shown below. Figure 5 The information details interface shown is 611.

[0227] The information details interface 611 is the interface for the information application, and it includes information content 612. Information content 612 includes the message content sent by the contact. The information details interface 611 also includes a microphone control 613, which can be used to trigger the electronic device 100 to collect the user's voice data through the microphone, convert the collected voice data into text information, and display it. The microphone control 613 can be a functional control provided by the information application, used to call the interface provided by the speech-to-text module 304, triggering the microcontroller unit 102 and the application processor 101 to implement the speech-to-text system function. Optionally, the information details interface 611 may also include one or more reply options, which can be used to trigger the electronic device 100 to send the message content corresponding to the reply option to the contact (e.g., "Okay, I have an appointment").

[0228] Step 503: Electronic device 100 can receive user input (e.g., click) to microphone control 613 and implement speech-to-text function through microcontroller 102 and application processor 101.

[0229] The microcontroller unit 102 of the electronic device 100 instructs the application processor 101 to convert the acquired voice data into text information. This step 503 includes multiple sub-steps, namely sub-steps 1 to 14, specifically, as follows... Figure 5 As indicated by the arrow in step 503: Sub-step 1: The touchscreen driver module 42 of the microcontroller unit 102 can detect the user's touch operation through the touch sensor of the touchscreen. The touchscreen driver module 42 can obtain touch operation information through the touch sensor. This touch operation information may include, but is not limited to, the position information of the screen touch point and the timestamp of the touch operation. Based on the timestamp of the touch operation, the touchscreen driver module 42 can obtain the touch event type. Here, the touch event type obtained by the touchscreen driver module 42 based on the touch operation information is a single click.

[0230] The touch screen driver module 42 can send the touch event type and screen touch point location information to the information application.

[0231] Optionally, since the microphone control 613 can only be used to trigger the speech-to-text system function, the touch screen driver module 42 can send only the position information of the screen touch point to the information application. The information application can determine to send the speech-to-text command to the application processor 101 based solely on the position information of the screen touch point.

[0232] Sub-step 2: The information application can determine the touch command used to trigger the operation corresponding to the microphone control 613 based on the touch event type and the location information of the screen touch point. That is, the touch command is used to instruct the application processor 101 to collect the user's voice through the microphone and convert the voice into text. Here, the touch command can be called a speech-to-text command. The information application can send the speech-to-text command to the speech-to-text module 304.

[0233] Understandably, the information application responds to input to the microphone control 613 by calling the interface provided by the speech-to-text module 304 to implement the speech-to-text function.

[0234] Optionally, the touchscreen driver module 42 can directly send the acquired touch operation information to the information application of the microcontroller unit 102. The information application can then generate voice-to-text commands based on the touch operation information.

[0235] Sub-step 3: The speech-to-text module 304 can send speech-to-text commands to the communication module 26.

[0236] Sub-step 4: Communication module 26 can send a voice-to-text command to communication module 22. Understandably, upon receiving the voice-to-text command, communication module 22 switches from sleep mode to non-sleep mode.

[0237] The descriptions of communication module 22 and communication module 26 can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0238] Sub-step 5: Communication module 22 can send voice-to-text commands to voice-to-text module 302.

[0239] Sub-step 6: The speech-to-text module 302 can notify the STT engine to perform speech-to-text operations based on the speech-to-text command.

[0240] Sub-step 7: The STT engine notifies the microphone driver module to collect the user's voice data.

[0241] Sub-step 8: The microphone driver module can collect the user's voice data through the microphone and return the collected voice data to the STT engine.

[0242] Sub-step 9: The STT engine can convert speech data into text information. For example, the STT engine may include a speech-to-text algorithm, which can be used to convert speech into text. The STT engine can then return the text information to the speech-to-text module 302.

[0243] Sub-step 10: The speech-to-text module 302 can send text information to the communication module 22.

[0244] Sub-step 11: Communication module 22 sends text information to communication module 26.

[0245] Sub-step 12: Communication module 26 can send text information to speech-to-text module 304.

[0246] Sub-step 13: The speech-to-text module 304 can send text information to the messaging application.

[0247] Sub-step 14: After receiving the text message, the information application can notify the image module to draw the interface of the information application that includes the text message.

[0248] In some examples, during the process of acquiring voice data and converting it into text information, the application processor 101 can send the converted text information to the microcontroller unit 102 every preset time interval (e.g., 0.5 seconds). Alternatively, the application processor 101 can send the text information to the microcontroller unit 102 after each converted character. In this way, the microcontroller unit 102 can display the text information input by the user's voice in real time.

[0249] In some examples, the application processor 101 may stop the speech-to-text operation if it does not receive voice data from the microphone driver module within a preset time (e.g., 5 seconds). The application processor 101 may also enter a sleep state after stopping the speech-to-text operation.

[0250] It is understood that the above-described triggering conditions for stopping the speech-to-text operation are merely examples. Embodiments of this application may also trigger the application processor 101 to stop performing the speech-to-text operation in other ways. For example, a messaging application may provide a close control that can trigger the microcontroller unit 102 to notify the application processor 101 to stop performing the speech-to-text operation; this embodiment of the application does not limit this approach.

[0251] In some examples, after the microphone control 613 is selected, the electronic device 100 can also de-display the microphone control 613 and display input prompt information, which can be used to prompt the user to input text via voice. For example, the input prompt information can be an animated prompt, such as the prompt icon 621 shown in stage 4. Optionally, the prompt icon 621 can be a pre-set animation.

[0252] The image module can send a request to the exercise application to obtain interface data provided by the application. Upon receiving this request, the exercise application can send the interface data back to the image module. The image module can receive the interface data sent by the exercise application and draw the exercise application interface 511 based on this data. Simultaneously, the image module can also draw at the distance icon 512 indicated by the screen touch point's position information. Figure 4 The marker 513 is shown. This marker 513 can be used to indicate that the distance icon 512 is marked.

[0253] Step 504: During the speech-to-text conversion process, the electronic device 100 can display text information 622. The text information 622 is text information obtained by the application processor 101 based on the speech data. For example, the text information 622 might include: "Okay, let's go to the newly opened one." It is understood that the content of the text information 622 can change according to the user's input speech data.

[0254] In this way, the electronic device 100 can draw the interface of the information application through the microcontroller unit 102 and obtain text information based on the user's voice data through the application processor 101.

[0255] It should be noted that the first application, an information application, is merely an example. The first application could also be other applications deployed on the microcontroller unit, such as a music application. Thus, when the microcontroller unit of the electronic device 100 is running a music application, it can obtain the song title input by the user's voice through the application processor.

[0256] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A processing method applied to an electronic device, characterized in that, The electronic device includes an application processor and a microcontroller unit; the method includes: The application processor controls the display screen to display a first interface, the first interface including one or more application icons, the one or more application icons including the icon of the first application; In response to a first input for an icon of the first application, the application processor retrieves the processor identifier of the first application from memory; When the processor identifier of the first application indicates that the microcontroller unit controls the display screen to show the interface of the first application, the application processor sends a first instruction to the microcontroller unit based on the processor identifier of the first application; After sending the first instruction to the microcontroller, the application processor switches to a sleep state; The microcontroller unit responds to the first instruction and controls the display screen to display the interface of the first application, the interface of the first application including the first control; The microcontroller unit sends a second instruction to the application processor in response to a second input to the first control. The application processor responds to the second instruction by switching from the sleep state to the non-sleep state and controlling the microphone to acquire first audio data; The application processor converts the first audio data into first text information; The application processor sends the first text information to the microcontroller unit; After sending the first text information to the microcontroller, the application processor switches to the sleep state; The microcontroller displays the first text information on the interface of the first application.

2. The method according to claim 1, characterized in that, The clock frequency of the microcontroller is lower than that of the application processor, and the operating current of the microcontroller is lower than that of the application processor, and the power consumption of the microcontroller is lower than that of the application processor.

3. A processing method applied to electronic devices, characterized in that, The electronic device includes an application processor and a microcontroller unit, both of which are configured in an accessibility mode; the method includes: The application processor controls the display screen to display a first interface, the first interface including one or more application icons, the one or more application icons including the icon of a second application; In response to a fourth input for the icon of the second application, the application processor retrieves the processor identifier of the second application from memory, the processor identifier of the second application being used to identify the interface of the second application being displayed on the display screen controlled by the microcontroller unit; The application processor sends a third instruction to the microcontroller unit based on the processor identifier of the second application, the third instruction including the identifier of the second application; After sending the third instruction to the microcontroller, the application processor switches to a sleep state; The microcontroller unit responds to the third instruction and controls the display screen to show the interface of the second application; The microcontroller controls the display screen to show the interface of the second application, and the first position of the interface of the second application includes second text information or a first control corresponding to the second text information. After receiving a third input for the first position, the microcontroller sends a fourth instruction to the application processor, the fourth instruction including the second text information; After receiving the fourth instruction, the application processor switches to a non-sleep state; The application processor, in response to the fourth instruction, converts the second text information into second audio data; The application processor controls the speaker to broadcast the second audio data.

4. The method according to claim 3, characterized in that, After the microcontroller receives a third input for the first position, the method further includes: Based on the touch information of the third input, the microcontroller determines that the third input is used to trigger the application processor to broadcast the second text information. The touch information of the third input includes the first position and the touch time of the third input.

5. The method according to claim 4, characterized in that, Based on the touch information from the third input, the microcontroller determines that the third input is used to trigger the application processor to read the second text information, specifically including: The microcontroller determines the touch event type of the third input based on the touch time of the third input; When the touch event type of the third input is the first event type, the microcontroller determines that the third input is used to trigger the electronic device to perform the operation of broadcasting the text information of the first location indication; The microcontroller unit acquires the second text information based on the first position; The microcontroller unit generates the fourth instruction.

6. The method according to claim 3, characterized in that, After the microcontroller receives a third input for the first position, the method further includes: The microcontroller sends first information to the application processor, the first information including a first location; Upon receiving the first information, the application processor switches to a non-sleep state; After determining, based on the first information, that the third input is used to trigger the application processor to broadcast the text information indicating the first location, the application processor sends a fifth instruction to the microcontroller unit. The fifth instruction is used to instruct the microcontroller unit to send the text information indicating the first location to the application processor.

7. The method according to claim 6, characterized in that, Before the microcontroller sends the fourth instruction to the application processor, the method further includes: After receiving the fifth instruction, the microcontroller obtains the second text information based on the first position; The microcontroller generates the fourth instruction based on the second text information.

8. The method according to claim 6 or 7, characterized in that, When the first information includes the touch information of the third input, the touch information of the third input includes the first position and the touch time of the third input; Based on the first information, the application processor determines the third input used to trigger the application processor to broadcast the text information indicating the first location, specifically including: The application processor determines the touch event type of the third input based on the touch time of the third input; When the touch event type of the third input is the first event type, the application processor determines that the third input is used to trigger the electronic device to perform the operation of broadcasting the text information of the first location indication.

9. The method according to claim 6 or 7, characterized in that, When the first information includes the touch event type of the third input and the first location; the application processor, based on the first information, determines the text information of the third input used to trigger the application processor to broadcast the first location indication, specifically including: When the touch event type of the third input is a first event type, the application processor determines that the third input is used to trigger the electronic device to perform the operation of broadcasting the text information indicating the first location.

10. The method according to claim 9, characterized in that, Before the microcontroller sends the first information to the application processor, the method further includes: The microcontroller determines the touch event type of the third input as the first event type based on the touch time in the touch information of the third input; The microcontroller obtains the first information based on the first event type and the first position in the touch information of the third input.

11. The method according to any one of claims 3-7, characterized in that, After the microcontroller sends the fourth instruction to the application processor, the method further includes: The microcontroller controls the display screen to display a first mark at the first position.

12. The method according to any one of claims 3-7, characterized in that, The clock frequency of the microcontroller is lower than that of the application processor, and the operating current of the microcontroller is lower than that of the application processor, and the power consumption of the microcontroller is lower than that of the application processor.

13. An electronic device, characterized in that, include: One or more processors and one or more memories; wherein the one or more processors include an application processor and a microcontroller unit, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when the one or more processors execute the computer instructions, cause the electronic device to perform the method as described in any one of claims 1-2 or 3-12.

14. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-2 or 3-12.

15. A chip system applied to an electronic device, the chip system comprising one or more processors, the one or more processors including an application processor and a microcontroller unit, the processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-2 or 3-12.

Images