Refresh rate switching method and electronic equipment

Abstract

The invention provides a refresh rate switching method and electronic equipment, relates to the technical field of terminal display, and is used for reducing the power consumption of the electronic equipment so as to prolong the endurance time and the service life of the electronic equipment. The method comprises the steps that when the electronic equipment starts a voice control function, the electronic equipment displays an interface of a first application at a first refresh rate in response to voice input by a user for opening the first application. And when the electronic equipment detects that the state of the electronic equipment meets a preset condition, that is, the electric quantity of the electronic equipment is greater than the preset electric quantity, and the moving state of the electronic equipment is a static state, the electronic equipment adopts a second refresh rate smaller than the first refresh rate to display the interface of the first application.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of terminal display technology, and more particularly to a refresh rate switching method and an electronic device. Background Art

[0002] With the rapid development of voice recognition technology, more and more electronic devices support voice interaction functions. By using the microphone of the electronic device to collect the voice commands issued by the user, human-computer interaction can be carried out with the electronic device, thereby freeing both hands and improving the efficiency of human-computer interaction.

[0003] When a user uses the voice interaction function of an electronic device, the microphone of the electronic device is required to receive the sound, convert the user's voice into text, understand and recognize the intention, and finally convert it into the corresponding voice command for execution. During this period, the power consumption of the electronic device will increase, causing the temperature of the electronic device to be too high. Over time, this will affect the battery life and service life of the electronic device. Summary of the invention

[0004] The embodiments of the present application provide a refresh rate switching method and an electronic device. In a scenario where a voice control function is activated in an electronic device, the refresh rate of a display screen is reduced in combination with the state of the electronic device to achieve the purpose of reducing the power consumption of the electronic device, thereby increasing the battery life and service life of the electronic device.

[0005] The embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a refresh rate switching method is provided, which is applied to an electronic device that supports a voice control function. The voice control function allows the user to receive a wake-up command without saying it. After receiving a voice command input by the user, the electronic device is controlled to perform an operation corresponding to the voice command. The wake-up command is different from the voice command. The wake-up command is used to wake up an application in the electronic device for voice interaction with the user.

[0007] When the electronic device starts the voice control function, in response to the user input of the voice to open the first application, the electronic device uses the first refresh rate to display the interface of the first application. When the electronic device detects that the state of the electronic device meets the preset condition, that is, the power of the electronic device is greater than the preset power, and the moving state of the electronic device is static, the electronic device uses a second refresh rate that is lower than the first refresh rate to display the interface of the first application.

[0008] According to the solution of the present application, in the scenario where the electronic device starts the voice control function, in response to the voice input by the user to open the first application, the electronic device uses the first refresh rate to display the interface of the first application; when the electronic device detects that the state of the electronic device meets the preset condition, that is, the power of the electronic device is greater than the preset power, and the mobile state of the electronic device is static, the second refresh rate is used to display the interface of the first application. Since the second refresh rate is lower than the first refresh rate, in the scenario where the electronic device starts the voice control function, combined with the state of the electronic device, the purpose of reducing the power consumption of the electronic device can be achieved by reducing the refresh rate, so as to improve the battery life and service life of the electronic device.

[0009] In addition, since the voice control function is a human-computer interaction with the electronic device through voice input by the user, rather than manual operation of the electronic device by the user, in the scenario where the voice control function of the electronic device is activated, when the power of the electronic device is greater than the preset power and the mobile state of the electronic device is stationary, lowering the refresh rate will not affect the user's experience.

[0010] In a possible implementation manner of the first aspect, the second refresh rate is a minimum refresh rate allowed when the electronic device runs the first application in the foreground.

[0011] In this way, as long as the normal display of the picture can be guaranteed, the refresh rate of the display screen can be reduced to the minimum as much as possible, so as to further reduce the power consumption of the electronic device.

[0012] In a possible implementation manner of the first aspect, the state of the electronic device satisfies the preset condition further includes that the distance between the display screen of the electronic device and the user's face is greater than a preset value.

[0013] In this implementation, when the distance between the display screen of the electronic device and the user's face is greater than a preset value, it means that the user is not holding the electronic device at this time, such as the electronic device can be placed on a stand or on a desktop. In this case, reducing the refresh rate of the display screen can achieve the purpose of reducing power consumption without affecting the user's experience.

[0014] In a possible implementation of the first aspect, an electronic device starts a voice control function, including: the electronic device displays a user interface for starting the voice control function, the user interface includes a preset switch; in response to a user operating the preset switch, the electronic device starts the voice control function.

[0015] Alternatively, the moving state of the electronic device is a stationary state, and the time length of the electronic device being in the stationary state is greater than a preset time length, and the electronic device activates the voice control function.

[0016] In a possible implementation of the first aspect, the interface of the first application is displayed using a second refresh rate, including: the electronic device obtains an application name of the first application; the electronic device determines an application type corresponding to the first application based on the first application name; the electronic device displays the interface of the first application using the second refresh rate based on the application type corresponding to the first application.

[0017] In a possible implementation of the first aspect, the electronic device uses a second refresh rate to display the interface of the first application according to the application type corresponding to the first application, including: if the application type corresponding to the first application is an audio and video application, and the first refresh rate is greater than the third refresh rate, the electronic device uses the second refresh rate to display the interface of the first application; wherein the third refresh rate is less than the first refresh rate and greater than the second refresh rate.

[0018] Alternatively, if the application type corresponding to the first application is an e-book application, the electronic device displays the interface of the first application at the second refresh rate.

[0019] In a possible implementation of the first aspect, the method also includes: the electronic device exits the voice control function, and the electronic device displays the interface of the first application using a first refresh rate; or, the state of the electronic device does not meet a preset condition, and the electronic device displays the interface of the first application using the first refresh rate.

[0020] In this implementation, when the electronic device exits the voice control function; or the state of the electronic device does not meet the preset conditions, the electronic device uses the first refresh rate to display the interface of the first application to improve the user experience.

[0021] In a possible implementation of the first aspect, the electronic device includes a voice control application APK, which is used to control the electronic device to perform a voice control function; after the electronic device starts the voice control function, the method also includes: the electronic device registers for listening to the application framework layer through the voice control APK of the application layer; when the electronic device opens the first application, the application framework layer of the electronic device callback notifies the voice control APK of the application layer of the current status of the electronic device and the application name of the first application.

[0022] In this implementation, the electronic device can register a monitor with the application framework layer through the voice control APK of the application layer to monitor the status of the electronic device and the application name of the first application running in the foreground, which can ensure the reliability of the refresh rate switching.

[0023] In a possible implementation of the first aspect, the method also includes: when the state of the electronic device does not meet the preset conditions, the electronic device cancels the registration and monitoring with the application framework layer through the voice control APK of the application layer; or, when the electronic device exits the voice control function, the electronic device cancels the registration and monitoring with the application framework layer through the voice control APK of the application layer.

[0024] In this implementation, when the state of the electronic device does not meet the preset conditions; or when the electronic device exits the voice control function, the electronic device cancels the registration monitoring with the application framework layer through the voice control APK of the application layer, which can further reduce the power consumption of the electronic device.

[0025] In a second aspect, an electronic device is provided, which has the function of implementing the refresh rate switching method described in any one of the first aspects, and the function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0026] According to a third aspect, an electronic device is provided, which supports a voice control function, wherein the voice control function allows a user to receive a wake-up instruction without saying a word, and after receiving a voice instruction input by the user, the electronic device is controlled to perform an operation corresponding to the voice instruction; the wake-up instruction is different from the voice instruction, and the wake-up instruction is used to wake up an application in the electronic device for voice interaction with the user; the electronic device comprises: a memory and one or more processors; the memory is used to store computer-executable instructions, and when the electronic device is running, the processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the refresh rate switching method described in any one of the above-mentioned first aspects.

[0027] In a fourth aspect, an electronic device is provided, which supports the voice control function described in the third aspect. The electronic device includes a processor; the processor is coupled to a memory, and after reading instructions in the memory, the processor executes the refresh rate switching method described in any one of the first aspects according to the instructions.

[0028] In a fifth aspect, a chip system is provided, which is applied to the electronic device described in the third aspect or the fourth aspect. The chip system includes a processor and an interface, wherein the interface is used to receive instructions and transmit them to at least one processor; at least one processor executes the instructions so that the electronic device executes the refresh rate switching method described in any one of the first aspects.

[0029] In a sixth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer can execute the refresh rate switching method described in any one of the above-mentioned first aspects.

[0030] In a seventh aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the refresh rate switching method described in any one of the above-mentioned first aspects.

[0031] Among them, the technical effects brought about by any design method in the second to seventh aspects can refer to the technical effects brought about by different design methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1A A schematic diagram of a scenario example of a method for controlling an electronic device by voice provided in an embodiment of the present application;

[0033] Figure 1B A schematic diagram of another scenario example of a method for controlling an electronic device by voice provided in an embodiment of the present application;

[0034] Figure 2 A schematic diagram of a scenario example of setting a preset switch for a voice control function provided in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of a scenario example of a mobile phone receiving user voice provided in an embodiment of the present application;

[0036] Figure 4 A schematic diagram of another scenario example of a mobile phone receiving user voice provided in an embodiment of the present application;

[0037] Figure 5 A schematic diagram of another scenario example of a mobile phone receiving user voice provided in an embodiment of the present application;

[0038] Figure 6 A schematic diagram of a scenario example of a mobile phone receiving user voice provided in another embodiment of the present application;

[0039] Figure 7 A schematic diagram of the principle of generating power consumption of an electronic device through a voice control function provided in an embodiment of the present application;

[0040] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0041] Fig. 9 A schematic diagram of a software framework of an electronic device provided in an embodiment of the present application;

[0042] Fig.10 A schematic diagram of a refresh rate switching method provided in an embodiment of the present application;

[0043] Fig.11A schematic flow chart of another refresh rate switching method provided in an embodiment of the present application;

[0044] Fig.12 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. The “and / or” in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0046] The following terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more.

[0047] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0048] Voice control function:

[0049] Many electronic devices support voice control functions. Electronic devices collect user voices through microphones, analyze and recognize user voices, and execute commands corresponding to user voices, so that users can control electronic devices through voice.

[0050] Generally speaking, in order to save power consumption of electronic devices and avoid false triggering, the voice control function needs to be turned on before it can be used. For example, the user inputs a preset word (called a wake-up word) to the electronic device through voice to wake up the electronic device. After the electronic device is awakened, it can execute the instructions corresponding to the user's voice, that is, the voice control function is turned on. For example, the user can turn on or off the voice control function by turning on or off the preset switch in the human-computer interaction interface of the electronic device.

[0051] In different electronic devices, the voice control function may have different names, such as "voice control", "intelligent voice", "voice assistant", "see and speak", "voice command", "free command", "intelligent AI", etc. The specific implementation of voice control functions with different names may also be different.

[0052] Several different implementations of the voice control function are described below by way of example.

[0053] Voice Assistant:

[0054] Before using a voice assistant to control an electronic device, the user needs to first input a wake-up word into the electronic device to wake up the electronic device. Generally speaking, before the electronic device is awakened, the microphone of the electronic device works in a power-saving mode (such as searching for signals at a lower power) to pick up sounds from the surrounding environment. The voice collected by the microphone is only detected for wake-up words at the kernel layer, and the corresponding recording channel of the voice assistant is not started in the system and driver of the electronic device.

[0055] In response to user operations (such as receiving a wake-up word input by the user's voice), the electronic device is awakened, and the corresponding recording channel of the voice assistant is started in the system and driver. After the electronic device is awakened, the voice (audio stream) collected by the microphone is sent to the voice assistant application for processing through the recording channel corresponding to the voice assistant. In this way, the electronic device can execute the instructions corresponding to the user's voice and realize the user's control of the electronic device through voice; the electronic device can also realize functions such as dialogue with the user.

[0056] For example, Figure 1A The figure shows a schematic diagram of a scenario in which a user uses a voice assistant to control a mobile phone. Figure 1A As shown, the mobile phone displays a desktop interface, and the user inputs a voice "Hello YOYO" into the mobile phone. In response to receiving the wake-up word "Hello YOYO", the mobile phone is awakened. Exemplarily, after the mobile phone is awakened, a voice "I am here" is played to prompt the user that the mobile phone has been awakened.

[0057] After the phone is awakened, the user can control the phone by voice. For example, Figure 1A As shown, the user inputs the voice "open video" to the mobile phone, and the mobile phone analyzes and recognizes the user's voice and executes the command corresponding to the voice "open video". Exemplarily, the mobile phone starts the video application in response to receiving the user's voice "open video".

[0058] For example, Figure 1B FIG. 1 shows another example of a scenario in which a user uses a voice assistant to control a mobile phone. Figure 1BAs shown, the mobile phone displays the desktop interface, and the user long presses the power button. In response to receiving the user's long press of the power button operation, the mobile phone is awakened. After the mobile phone is awakened, the user can control the mobile phone by voice. For example, Figure 1B As shown, the user inputs the voice "open video" to the mobile phone, and the mobile phone analyzes and recognizes the user's voice and executes the command corresponding to the voice "open video". Exemplarily, the mobile phone starts the video application in response to receiving the user's voice "open video".

[0059] In some implementations, after the electronic device is awakened, the user can issue a command to the electronic device by inputting voice into the electronic device, and the electronic device executes the command corresponding to the voice. After the electronic device executes a command, or if the electronic device does not receive a command from the user through voice within a period of time (for example, within 8 seconds) after being awakened, the electronic device no longer responds to the command issued by the user through voice. For example, the electronic device will turn off the recording channel corresponding to the voice assistant. The user needs to input the wake-up word into the electronic device again to wake up the electronic device before he can issue a command to the electronic device by inputting voice into the electronic device again. In other words, after the electronic device is awakened, it enters a "short reception" state and can respond to the user's voice command within a short period of time (for example, within 8 seconds).

[0060] In some implementations, when the electronic device is connected to the network, it supports entering a continuous conversation scenario after being awakened, and the user can have a continuous conversation with the electronic device. After each broadcast, the electronic device will continue to pick up the sound without repeated awakening until the user exits the continuous conversation through instructions such as "exit".

[0061] It can be seen that:

[0062] It can be seen that it is implemented locally by electronic devices and does not require a network connection.

[0063] Visible and Talkable (or Visible and Talkable) is controlled by a preset switch. Users can turn on Visible and Talkable by turning on the preset switch, and can also exit Visible and Talkable by turning off the preset switch.

[0064] When the preset switch corresponding to Visible is Talk is turned on, the corresponding recording channel of Visible is Talk is started in the system and driver of the electronic device. In this way, the voice (audio stream) collected by the microphone can be sent to the Visible is Talk application for processing through the corresponding recording channel of Visible is Talk, so that the user can control the electronic device through voice.

[0065] It can be seen that after being turned on, the electronic device enters a "long receiving" state. The user can give instructions to the electronic device through voice at any time without entering a wake-up word to wake up the electronic device.

[0066] Exemplary, reference Figure 3 , the mobile phone displays the desktop interface, the user inputs the voice "open video" into the mobile phone, the mobile phone analyzes and recognizes the user's voice, and executes the command corresponding to the voice "open video". Exemplarily, the mobile phone starts the video application in response to receiving the user's voice "open video".

[0067] Hot words:

[0068] The electronic device can be set with some hot words, that is, after it is turned on, if the received user voice matches at least one of the set hot words, the electronic device executes the instruction corresponding to the user voice.

[0069] The hot words may be pre-configured on the electronic device, or obtained based on the content on the display interface of the electronic device, or input by the user, etc.

[0070] In the scenario where the user's hands are occupied, voice is the most important and fastest way of human-computer interaction. The electronic device responds to the wake-up command spoken by the user (such as the wake-up word "Hello, YOYO") to achieve interaction between the user and the electronic device. Usually, the electronic device is installed with an application that can interact with the user by voice (such as a voice assistant). The user can interact with the electronic device by voice through the voice assistant to achieve functions that require multiple manual operations to achieve.

[0071] Among them, the scenarios in which the user's hands are occupied mainly include: the user is cooking in the kitchen and using an electronic device to watch videos; or the user is washing and using an electronic device to watch videos or listen to music; or the user is eating on the sofa in the living room and using an electronic device to watch videos; or the user is resting in the bedroom and the electronic device is in the living room and the alarm is ringing.

[0072] In each of the above scenarios, users can use the voice assistant to watch videos (such as pause, play, next / previous, increase / decrease volume, etc.), as well as turn off the alarm / remind later and other operations.

[0073] Currently, when using a voice assistant, the user usually needs to say a wake-up word to wake up the voice assistant. After waking up the voice assistant, the user can continue to speak voice to control the corresponding operation of the electronic device. For example, in response to the user saying the wake-up word "Hello, YOYO", the electronic device wakes up the voice assistant. Then, in response to the user saying "Play next", the electronic device can perform the operation of playing the next video.

[0074] In some embodiments, after the electronic device wakes up the voice assistant once and performs an operation corresponding to a voice command, the electronic device automatically turns off the voice assistant. When the user needs to use the voice assistant for a long time for voice interaction, the electronic device needs to frequently respond to the wake-up word spoken by the user to wake up the voice assistant. Frequent wake-up of the voice assistant by the electronic device will inevitably cause higher power consumption, and frequent wake-up of the voice assistant cannot meet the diverse needs of users.

[0075] Based on the above analysis, an embodiment of the present application provides an electronic device that supports a voice control function. The voice control function allows a user to directly speak a voice command without saying a wake-up word, and then control the electronic device to perform a voice interaction service corresponding to the voice command.

[0076] In one example, the voice control function needs to be turned on before it can be used. For example, the user can turn on or off the voice control function by turning on or off a preset switch in the human-computer interaction interface of the electronic device. In different electronic devices, the voice control function may have different names, such as "voice control", "smart voice", "voice command", "smart artificial intelligence (AI)", etc.

[0077] Exemplarily, the voice control function supported by the electronic device may integrate multiple voice interaction services, and the multiple voice interaction services may be integrated into the voice application package (Android application package, APK) of the electronic device. For example, the multiple voice interaction services may have different names, such as "visible and audible", "free command", "AI subtitles", etc.

[0078] Among them, after each of the above-mentioned multiple voice interaction services is started, the electronic device implements different functions based on the received user's voice. For example, when Visible and Talkable is started, the electronic device can implement various operating functions in third-party applications (such as long video applications, short video applications, etc.) through the received user's voice. When the free command is started, the electronic device can implement various operating functions in system applications (such as alarm clock applications, phone applications, etc.) through the received user's voice. When AI subtitles are started, the electronic device can recognize the user's voice and automatically generate synchronized subtitles.

[0079] In one example, for the above-mentioned multiple voice interaction services, each voice interaction service supports different types of instructions and hot words. After any voice interaction service is started, the electronic device receives the voice input by the user. If the user's voice received by the electronic device matches the instruction or hot word supported by the voice interaction service, the electronic device performs the operation corresponding to the user's voice.

[0080] Among them, the above instructions support generalization. For example, different users have different voices, but the operations that need to be performed by the electronic device are the same. For example, the user's voice is "Like / Like", but the operation performed by the electronic device is "Like". Correspondingly, hot words involve some relatively fixed texts, such as the application name in the application list, the text displayed in the permission pop-up window, and some text on the page. For example, when the user's voice matches the hot word, the electronic device can execute the instruction corresponding to the user's voice.

[0081] It should be noted that the electronic device can understand the semantics of the user's voice. That is, the user's voice may not be completely consistent with the hot word, as long as the semantics are the same or close. That is, the user's voice and the hot word match, but they do not have to be completely consistent. For example, in Figure 3 In the scenario shown, the user inputs the voice "open video", the system hot words include "video", and "open video" and "video" have the same semantics, that is, the two match.

[0082] Taking the example of launching Visible and Talkable on an electronic device, illustratively, Visible and Talkable supports system instructions, short video instructions, long video instructions, etc., as shown in Table 1 below.

[0083] Table 1

[0084]

[0085] It should be noted that Table 1 is only an example of the instructions supported by Visible and Sayable in the embodiment of the present application. Of course, Visible and Sayable can also support other instructions, which are not listed here.

[0086] Exemplarily, the embodiment of the present application is introduced by taking the activation of “Visible and Speakable” as an example.

[0087] Among them, Visible and Sayable has corresponding function entrances in electronic devices, such as control center, setting application, etc. Users can enter the corresponding function entrance (such as setting application) and turn on or off the Visible and Sayable function in the human-computer interaction interface of the electronic device by turning on or off the preset switch.

[0088] Taking the above electronic device as a mobile phone, for example, Figure 2As shown, the user can open the settings function of the mobile phone; for example, the user clicks on the application icon of the "Settings" application on the desktop. In response to the user clicking on the application icon of the "Settings" application, the mobile phone displays the "Settings" interface 101. The "Settings" interface 101 includes a "Smart Voice" option 102, and the "Smart Voice" option 102 is used to set the smart voice function. Exemplarily, in response to the user clicking on the "Smart Voice" option 102, the mobile phone displays the "Smart Voice" interface 103, and the "Smart Voice" interface 103 includes a "Visible and Speakable" option 104. The user can click on the "Visible and Speakable" option 104 to set visible and speakable related options. Exemplarily, refer to Figure 2 In response to the user clicking on the “visible and talkable” option 104 , the mobile phone displays the “visible and talkable” interface 105 .

[0089] Exemplarily, the "visible and talkable" interface 105 includes a prompt page 106, which includes prompt information for prompting the user how to use the visible and talkable function. The "visible and talkable" interface 105 also includes a "voice control" switch. When the "voice control" switch is turned on, the mobile phone starts the visible and talkable function; when the "voice control" switch is turned off, the mobile phone exits the visible and talkable function.

[0090] In one example, after Visible and Talkable is turned on, the mobile phone displays a first prompt icon, and the first prompt icon is used to prompt that Visible and Talkable is turned on. Figure 2 As shown, the status bar of the mobile phone display interface displays a prompt icon 10a. In actual implementation, the prompt icon 10a can be a microphone reception icon, which is used to indicate that the mobile phone enters a continuous reception state.

[0091] After the Visible and Talkable function is turned on, the electronic device enters a continuous sound receiving state, and the microphone of the electronic device will continuously collect surrounding sounds. Optionally, in one implementation, after any function on the electronic device turns on the recording channel of the terminal device, the terminal device will send a prompt message to the user to remind the user that the terminal device is in a sound receiving state. In this way, the user's privacy can be prevented from being leaked. For example, after the Visible and Talkable function is turned on, the electronic device turns on the recording channel and enters a continuous sound receiving state, and a second prompt icon is displayed on the display interface of the terminal device, and the second prompt icon is used to prompt that the microphone is recording sound. For example, Figure 2 As shown, the status bar of the mobile phone display interface displays a prompt icon 10b.

[0092] After the visible and speakable function is turned on, the electronic device enters a continuous sound receiving state. The user can input voice into the electronic device, and the electronic device analyzes and recognizes the voice input by the user. If the user's voice is recognized to match any page text (i.e., hot words), the operation corresponding to the voice is performed.

[0093] In one example, after Visible & Speaking is started, the electronic device can support the start and exit of applications, switching of desktops, etc. through voice. Also, after the electronic device starts an application, various operating functions in the application can be controlled through voice.

[0094] Exemplarily, the following describes four scenarios to illustrate how to control an electronic device to perform corresponding operations through voice after Visible and Speaking is activated.

[0095] Scenario 1: After Visible & Speaking is started, use voice control to start the video application on the electronic device.

[0096] Scenario 2: After Kankeshuo is started and a short video application is running on an electronic device, various operating functions of the short video application (such as switching, liking, collecting, following, etc.) are controlled by voice.

[0097] Scenario 3: After Visible & Speaking is started and an e-book application is running on the electronic device, various operating functions of the e-book application (such as turning pages of a novel, adding to a bookshelf, etc.) are controlled by voice.

[0098] Scenario 4: After Visible & Talkable is started and a news application is running on the electronic device, various operating functions of the news application (such as page sliding, entering a tag, etc.) are controlled by voice.

[0099] For example, Figure 3 A schematic diagram of a scenario example in which a mobile phone receives user voice in scenario 1 is shown.

[0100] refer to Figure 3 , the mobile phone displays the desktop interface 111. Hot words corresponding to the desktop interface 111 include: "swipe up / down / left / right", "return", "return to desktop", "open application (such as open video, open game)", etc. The user inputs the voice "open video" to the mobile phone, and the mobile phone receives the voice "open video" input by the user, which matches the hot word "open application". In response to the received voice "open video", the mobile phone executes the instruction corresponding to the voice "open video", the mobile phone starts the video application, and displays the user interface 112 of the video application.

[0101] For example, Figure 4 A schematic diagram of a scenario example in which a mobile phone receives user voice in scenario 2 is shown.

[0102] refer to Figure 4, the mobile phone displays the playback interface 113 of the short video application. The hot words corresponding to the short video application include: "play", "pause", "stop", "fast forward", "fast rewind", etc. The user inputs the voice "pause" to the mobile phone. The voice "pause" input by the user is received, and the voice "pause" matches the hot word "pause". In response to receiving the voice "pause", the mobile phone executes the instruction corresponding to the voice "pause" to pause the playback of the short video. In one example, Figure 4 As shown, a "play" button 114 is displayed on the playback interface 113 of the short video application to prompt the user that the short video has been paused. The mobile phone can continue to play the short video in response to the user clicking the "play" button 114. The mobile phone can also continue to play the short video in response to the user inputting a voice "play".

[0103] For example, Figure 5 A schematic diagram of a scenario example in which a mobile phone receives user voice in scenario three is shown.

[0104] refer to Figure 5 , the mobile phone displays the application interface 115 of the e-book application. The hot words corresponding to the e-book application include: "previous page", "next page", "table of contents", "next chapter", etc. The user inputs the voice "next page" into the mobile phone. The voice "next page" input by the user is received, and the voice "next page" matches the hot word "next page". In response to receiving the voice "next page", the mobile phone executes the instruction corresponding to the voice "next page" and displays the application interface 116 of the e-book application. Among them, the page number of application interface 116 is page 20, the page number of application interface 115 is page 19, and application interface 116 is the next page of application interface 115.

[0105] For example, Figure 6 A schematic diagram of a scenario example in which a mobile phone receives user voice in scenario 4 is shown.

[0106] refer to Figure 6 , the mobile phone displays an application interface 117 of the news application. Hot words corresponding to the news application include: "swipe left", "swipe right", "swipe up", "swipe down", etc. The user inputs a voice "swipe left" to the mobile phone. The voice "swipe left" input by the user is received, and the voice matches the hot word "swipe left". In response to the received voice "swipe left", the mobile phone executes the instruction corresponding to "swipe left" and displays the application interface 118 of the news application.

[0107] In combination with the above embodiment, after Visible and Talkable is started, the microphone of the electronic device is in a continuous sound receiving state, and the electronic device needs to convert the user's voice into text for intention understanding and recognition, and finally convert it into corresponding instructions for execution. Figure 7As shown, in the process of the electronic device converting the user's voice into corresponding instructions, the electronic device also performs echo cancellation operations, page monitoring (obtaining hot words) operations, and displaying user interfaces (such as the first display icon, floating capsule display, etc.) and other operations.

[0108] For example, Figure 7 As shown, echo cancellation is used to eliminate the sound emitted by the speaker of the electronic device to avoid interfering with the microphone collecting the user's voice. The voice conversion process may include: automatic speech recognition (ASR); natural language understanding (NLU); dialogue management (DM) function implementation. Among them, ASR is used to convert voice signals into text information; NLU is used to convert text information into semantics that can be understood by electronic devices, and understand the intention of the text information based on the semantics, and DM is used to generate instructions that the electronic device finally executes based on the understood intention of the text information.

[0109] exist Figure 7 The process shown will undoubtedly increase the power consumption of the electronic device, causing the temperature of the electronic device to be too high. And as time goes by, it will also affect the battery life and service life of the electronic device. Based on this, on the basis of the visible and tangible startup of the electronic device, how to reduce the power consumption of the electronic device and increase the battery life and service life of the electronic device has become a technical problem that needs to be solved urgently.

[0110] The embodiment of the present application provides a refresh rate switching method, which can reduce the refresh rate of the display screen in a scenario where the electronic device is activated and visible and can be spoken, combined with the status information of the electronic device (such as static state, power mode, etc.), thereby achieving the purpose of reducing the power consumption of the electronic device and improving the battery life and service life of the electronic device.

[0111] In conventional technology, electronic devices use different refresh rates (i.e., the refresh rate of the display screen) when running different applications. For example, when an electronic device runs an audio or video application, the refresh rate used by the electronic device is 60Hz. When an electronic device runs a desktop application, the refresh rate used by the electronic device is 120Hz. When an electronic device runs a game application, the refresh rate used by the electronic device is 144Hz, etc.

[0112] In the above examples, the refresh rate used by the electronic device when running different applications can be understood as the default refresh rate set by the device manufacturer (or user). In order not to affect the user experience, the default refresh rate set by the device manufacturer (or user) matches the output frame rate (unit: FPS) of the application.

[0113] The output frame rate is the number of frames per second that the application (i.e., the application) outputs to the display. The default refresh rate matches the output frame rate of the application, which means that the default refresh rate is greater than or equal to the output frame rate. For example, for audio and video applications, the default refresh rate is 60Hz and the output frame rate is 60FPS.

[0114] It should be noted that the reason why the default refresh rate is greater than or equal to the output frame rate is that when the default refresh rate is lower than the output frame rate, the display screen cannot display all the images output by the application per second. Therefore, the electronic device will discard part of the images output by the application per second or merge part of the images, causing the image to freeze and affecting the user experience.

[0115] In this way, the default refresh rate set by the device manufacturer (or user) is generally high (such as greater than 60Hz). In Visible and Speakable, the display screen does not need a high refresh rate. This is because Visible and Speakable is a human-computer interaction with the electronic device through the voice input of the user, rather than the user manually operating the electronic device. Therefore, when the electronic device starts Visible and Speakable, combined with the status information of the electronic device, the refresh rate of the display screen is reduced. This can achieve the purpose of reducing the power consumption of the electronic device without affecting the user experience, thereby increasing the battery life and service life of the electronic device.

[0116] The refresh rate switching method provided in the embodiment of the present application is for applications applicable to Visible and Sayable. Some refresh rates will be pre-configured in the electronic device, and these refresh rates can adapt to the application running in the foreground of the electronic device when the electronic device starts Visible and Sayable. It can be understood that these refresh rates are less than the above-mentioned default refresh rate. In other words, on the electronic device, part of the refresh rate comes from the refresh rate with a higher value pre-configured by the device manufacturer (or user) (hereinafter referred to as refresh rate A, or the first refresh rate). This part of the refresh rate is used to adapt the application running in the foreground of the electronic device when the electronic device does not start Visible and Sayable; the other part is a refresh rate with a lower value pre-configured (hereinafter referred to as refresh rate B, or the second refresh rate). This other part of the refresh rate is when the electronic device starts Visible and Sayable, adapting the application running in the foreground of the electronic device.

[0117] In an example, the electronic device pre-configures a correspondence between the application type and the refresh rate A. Exemplarily, the correspondence between the application type and the refresh rate A is shown in Table 2.

[0118] Table 2

[0119] Application Type Refresh rateA Audio Applications A1 Short video applications A2 Long video applications A3 E-book application A4 Home screen management A5

[0120] In another example, the electronic device pre-configures a correspondence between the application type and the refresh rate B. Exemplarily, the correspondence between the application type and the refresh rate B is shown in Table 3.

[0121] Table 3

[0122]

[0123]

[0124] It should be noted that the above Tables 2 and 3 are only illustrative of the correspondence between application types and refresh rates. In other embodiments, the electronic device may obtain the correspondence between application types and refresh rates in other ways; for example, the correspondence between application types and refresh rates may not be in the form of a configuration table; for example, the correspondence between application types and refresh rates may be different from that in the above examples. In other embodiments, the electronic device may not save the correspondence between application types and refresh rates; for example, the electronic device may obtain the correspondence between application types and refresh rates from a server. The embodiments of the present application do not limit the specific manner in which the electronic device obtains the refresh rate corresponding to the application type.

[0125] Of course, the electronic device can also directly configure the correspondence between the application name (such as the application package name) and the refresh rate. Please refer to the above embodiment, which will not be listed here.

[0126] In an example, the electronic device can determine the application type according to the application name. Exemplarily, the electronic device is pre-configured with a correspondence between the application name and the application type. For example, the correspondence between the application type and the application name is shown in Table 4.

[0127] Table 4

[0128]

[0129] It is understandable that in another example, the division of application types may be different from the above example; for example, more or fewer application types may be divided. The application names included in each application type may also be adjusted according to actual conditions.

[0130] When any application on an electronic device is running in the foreground, the electronic device can match the corresponding application type based on the application name of the application, and then determine the corresponding refresh rate based on the matched application type, and control the display screen to display the display interface of the application at the refresh rate.

[0131] For example, in a scenario where the electronic device is not visible or talkable, the electronic device starts application 1 (or the first application), that is, application 1 runs in the foreground. The electronic device starts application 1 according to the application name (such as ), determine that the application type corresponding to application 1 is an audio application. Then, the electronic device determines the refresh rate of the display screen as refresh rate A1 according to the application type, and the electronic device controls the display screen to display the display interface of application 1 at refresh rate A1.

[0132] In the scenario where the electronic device starts the visible and talkable state, the electronic device starts application 1, that is, application 1 runs in the foreground. The electronic device starts application 1 according to the application name (such as ), determine that the application type corresponding to application 1 is an audio application. Then, the electronic device determines the refresh rate of the display screen as refresh rate B1 according to the application type, and the electronic device controls the display screen to display the display interface of application 1 at refresh rate B1.

[0133] In one example, the refresh rate A1 may be, for example, 90 Hz, and the refresh rate B2 may be, for example, 60 Hz, that is, the refresh rate B2 is less than the refresh rate A1, and the refresh rate B2 is the minimum refresh rate allowed when the electronic device runs application 1 in the foreground.

[0134] In another exemplary embodiment, in a scenario where the electronic device is not visible or talkable, the electronic device starts application 1, that is, application 1 runs in the foreground. The electronic device starts application 1 according to the application name (such as ), determine that the application type corresponding to application 1 is an e-book application. Then, the electronic device determines the refresh rate of the display screen as refresh rate A4 according to the application type, and the electronic device controls the display screen to display the display interface of application 1 at refresh rate A4.

[0135] In the scenario where the electronic device starts the visible and talkable state, the electronic device starts application 1, that is, application 1 runs in the foreground. The electronic device starts application 1 according to the application name (such as ), determine that the application type corresponding to application 1 is an e-book application. Then, the electronic device determines the refresh rate of the display screen as refresh rate B4 according to the application type, and the electronic device controls the display screen to display the display interface of application 1 at refresh rate B4.

[0136] In another example, the refresh rate A4 may be, for example, 60 Hz, and the refresh rate B4 may be, for example, 30 Hz, that is, the refresh rate B4 is less than the refresh rate A4, and the refresh rate B4 is the minimum refresh rate allowed when the electronic device runs application 1 in the foreground.

[0137] In addition, in the embodiment of the present application, for application 1 running in the foreground of the electronic device, in the scenario where the electronic device does not start the visible and talkable function, the electronic device controls the display screen to display the display screen of application 1 at a refresh rate A. In the scenario where the electronic device starts the visible and talkable function, the electronic device controls the display screen to display the display screen of application 1 at a refresh rate B. That is to say, for the same application, when the visible and talkable function is not started, the refresh rate of the display screen is refresh rate A; when the visible and talkable function is started, the refresh rate of the display screen is reduced from refresh rate A to refresh rate B.

[0138] In this way, when the electronic device starts to be visible and talkable, the electronic device can reduce the power consumption of the electronic device by lowering the refresh rate.

[0139] In one example, the electronic device can reduce the refresh rate of the display screen to the minimum refresh rate allowed by the application running in the foreground of the electronic device. In other words, as long as the picture can be displayed normally, the refresh rate of the display screen can be reduced to the minimum as much as possible.

[0140] In some embodiments, after the electronic device determines the application type corresponding to application 1, if the application type corresponding to application 1 is an audio application and the refresh rate A is greater than the refresh rate C (or the third refresh rate, such as 60 Hz), the electronic device uses the refresh rate B to display the interface of application 1. If the refresh rate A is less than the refresh rate C, the electronic device uses the refresh rate C to display the interface of application 1.

[0141] In addition, if the application type corresponding to application 1 is an e-book application, the electronic device uses refresh rate B to display the interface of application 1.

[0142] When the electronic device starts to be visible and talkable, the electronic device can also monitor in real time whether the electronic device has application switching. When the electronic device monitors application switching, the electronic device re-determines the corresponding refresh rate based on the application running in the foreground, and controls the display screen to display the display interface of the application at the refresh rate.

[0143] For example, in the scenario where the electronic device starts to be visible and talkable, when the electronic device detects the switch from application 1 to application 2 (or the second application), application 2 is running in the foreground. The electronic device determines the application type corresponding to application 2 as main screen management based on the application name of application 2 (such as desktop application). Then, the electronic device determines the refresh rate of the display screen as refresh rate B5 based on the application type, and the electronic device controls the display screen to display the display interface of application 2 at refresh rate B5.

[0144] It is understandable that in the embodiment of the present application, when the electronic device does not start the visible and talkable state, the electronic device starts application 2, that is, the application runs in the foreground. The refresh rate determined by the electronic device according to the application name of application 2 is refresh rate A5, and the refresh rate A5 is greater than refresh rate B5. Exemplarily, the above refresh rate A5 can be, for example, 120Hz, and the refresh rate B5 can be, for example, 60Hz.

[0145] In some embodiments, when the electronic device is in a visible and talkable scenario, by detecting the status information of the electronic device, when the status information of the electronic device meets the preset conditions, the electronic device reduces the refresh rate of the display screen to the above-mentioned refresh rate B. This can reduce the power consumption of the electronic device without affecting the user experience, thereby improving the battery life and service life of the electronic device.

[0146] Exemplarily, the state information of the electronic device includes at least: movement state and power information. The movement state includes static state and moving state. The preset conditions include: the movement state of the electronic device is static state, and the current power of the electronic device is greater than the preset power.

[0147] It should be noted that there is no specific limitation on the preset power level, which is subject to the actual setting. Generally, the power level of an electronic device is generally expressed as 1% to 100%, on this basis, the preset power level may be, for example, 20%.

[0148] Based on the above scheme, in the scenario where the electronic device starts the visible and talkable function, when the mobile state of the electronic device is stationary and the current power of the electronic device is greater than the preset power, the electronic device reduces the refresh rate of the display screen to refresh rate B based on the application running in the foreground, and controls the display screen to display the display interface of the application at refresh rate B. Since the visible and talkable function is a human-computer interaction with the electronic device through the voice input by the user, rather than the user manually operating the electronic device, in the visible and talkable scenario, reducing the refresh rate of the display screen can achieve the purpose of reducing the power consumption of the electronic device without affecting the user experience, thereby increasing the battery life and service life of the electronic device.

[0149] In addition, in the above scheme, the reason why the preset condition includes that the current power of the electronic device is greater than the preset power is that when the power of the electronic device is low (such as less than 20%), the refresh rate of the display screen will be adjusted by default. In this case, the power consumption of the electronic device will increase and the battery life of the electronic device will be reduced. Therefore, in the embodiment of the present application, the refresh rate of the display screen is reduced only when the mobile state of the electronic device is stationary and the power is greater than the preset power, so as to achieve the purpose of reducing the power consumption of the electronic device.

[0150] In one example, the state information further includes the distance between the display screen of the electronic device and the user's face. The preset condition also includes that the distance between the display screen of the electronic device and the user's face is greater than a preset value.

[0151] It should be noted that, when the mobile state of the electronic device is a stationary state, the distance between the display screen of the electronic device and the user's face is greater than a preset value, which can be understood as: the user is not holding the electronic device, such as the electronic device may be placed on a bracket, or the electronic device is placed on a desktop, etc.

[0152] In this way, when the mobile state of the electronic device is stationary, the distance between the display screen of the electronic device and the user's face is greater than a preset value, and the power of the electronic device is greater than a preset power, the refresh rate of the display screen can be reduced to a minimum without losing the user experience, thereby achieving the purpose of reducing the power consumption of the electronic device.

[0153] In some embodiments, when the electronic device starts the visible and talkable scene, when the status information of the electronic device does not meet the preset conditions, the electronic device restores the refresh rate of the display screen to the default refresh rate, that is, the electronic device switches the refresh rate of the display screen from refresh rate B to refresh rate A. In other embodiments, when the electronic device exits the visible and talkable state, the electronic device switches the refresh rate of the display screen from refresh rate B to refresh rate A.

[0154] Exemplarily, the status information of the electronic device not satisfying the preset conditions includes: the movement state of the electronic device switches from a stationary state to a moving state; or, the power of the electronic device is less than a preset power; or, the distance between the display screen of the electronic device and the user's face is less than a preset value.

[0155] The refresh rate switching method provided in the embodiment of the present application can be applied to electronic devices that support voice control functions. The above electronic devices may include mobile phones, tablet computers, laptops, personal computers (PCs), ultra-mobile personal computers (UMPCs), handheld computers, netbooks, smart home devices (e.g., smart TVs, smart screens, large screens, smart speakers, smart air conditioners, etc.), personal digital assistants (PDAs), wearable devices (e.g., smart watches, smart bracelets, etc.), vehicle-mounted devices, virtual reality devices, etc., and the embodiment of the present application does not impose any restrictions on this.

[0156] For example, Figure 8 1 is a schematic diagram of the structure of an electronic device 100. The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195.

[0157] Among them, the above-mentioned sensor 180 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor, bone conduction sensor and gravity sensor.

[0158] Exemplarily, the electronic device 100 can detect the state information of the electronic device 100 through the sensor 180. For example, the movement state of the electronic device and the distance between the display screen of the electronic device and the user's face can be detected through a gyroscope sensor, an acceleration sensor, a gravity sensor, a distance sensor, etc.

[0159] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0160] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0161] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0162] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0163] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0164] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0165] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of an electronic device. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0166] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0167] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0168] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0169] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-LED, Micro-OLED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc.

[0170] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0171] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0172] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0173] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals. For example, when an electronic device selects a frequency point, a digital signal processor is used to perform Fourier transform on the frequency point energy.

[0174] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. In this way, the electronic device can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0175] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously self-learn. NPU can realize applications such as intelligent cognition of electronic devices, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0176] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0177] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 are arranged in the processor 110. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" and the "microphone", is used to convert sound signals into electrical signals. The headphone interface 170D is used to connect wired headphones. The headphone interface 170D can be a USB interface 130, or it can be a 3.5mm open mobile electronic device platform (open mobile terminal platform, OMTP) standard interface, the cellular telecommunications industry association of the USA (cellular telecommunications industry association of the USA, CTIA) standard interface. In the embodiment of the present application, after the voice control function is turned on, the electronic device 100 continuously collects surrounding sounds through the microphone 170C.

[0178] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, audio, video and other files are stored in the external memory card.

[0179] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121. For example, in an embodiment of the present application, the processor 110 can execute instructions stored in the internal memory 121, and the internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area may store data (such as video files) created during the use of the electronic device, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0180] The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 195, or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device. The electronic device can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0181] In the embodiment of the present application, the electronic device 100 is an electronic device that can run an operating system and install application programs. For example, the operating system running on the electronic device 100 can be Android system, system, System, etc.

[0182] In some embodiments, the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplarily described.

[0183] Fig. 9 It is a software structure block diagram of the electronic device according to an embodiment of the present application.

[0184] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. The system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime and system library, and kernel layer.

[0185] The application layer may include a series of application packages APK. For example, camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications. In the embodiment of the present application, the application layer includes a voice control APK, which is used to provide a voice control function for the electronic device. Among them, the voice control APK is divided into a user interface (UI) layer and a logic layer. The UI layer is used to display interface information related to the voice control APK on the user interface of the electronic device; for example, the UI layer may include a service presentation module for displaying the above-mentioned FIG1- Figure 6The logic layer is used to process functions related to the voice control APK, such as switching of refresh rates, distribution of commands, and processing of user voice.

[0186] For example, Fig. 9 As shown, the logic layer includes: a screen element parsing module, an instruction distribution module, etc. Among them, the screen element parsing module parses the elements on the display interface of the electronic device, such as controls, text on controls, etc. The instruction distribution is used to execute the process of matching elements on the display interface according to the instructions.

[0187] The logic layer also includes a voice processing engine for recognizing and processing voice and generating instructions that are ultimately executed by the electronic device.

[0188] For example, Fig. 9 As shown, the logic layer also includes a registration monitoring module, a state management module, a scene perception module, and a refresh rate switching module. Among them, the registration monitoring module is used to monitor the power, application switching, and mobile status of the electronic device. The state management module is used to determine whether the state information of the electronic device meets the preset conditions. The scene perception module is used to match the corresponding application type according to the application running in the foreground of the electronic device. The refresh rate switching module is used to determine the refresh rate according to the application type when the state information of the electronic device meets the preset conditions.

[0189] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer may include some predefined functions.

[0190] like Fig. 9 As shown, the application framework layer may include a window manager, a content provider, a view system, a resource manager, an activity manager service (AMS), an application manager (package manager service, PMS), a screen element acquisition module, a multi-mode interaction module, and a refresh rate control module, etc.

[0191] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0192] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0193] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0194] The resource manager provides various resources to the application layer, such as localized strings, icons, images, layout files, video files, and so on.

[0195] AMS is mainly responsible for the startup, switching, and scheduling of the four major components in the system, as well as the management and scheduling of application processes. Its responsibilities are similar to the process management and scheduling module in the operating system. When a process or component is started, the request will be passed to AMS through the Binder communication mechanism, and AMS will then handle it uniformly.

[0196] PMS handles package management related tasks, such as application installation, application uninstallation, etc. It is also used to provide all the information of an application, such as startup, shutdown, etc.

[0197] The screen element acquisition module is used to acquire elements on the display interface of the electronic device, such as controls, text on controls, etc.

[0198] The multi-mode interaction module is used to manage the execution of voice commands. For example, the package name of the application running in the foreground of the electronic device is obtained, and the voice command is sent to the application so that the application executes the command.

[0199] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0200] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0201] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0202] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0203] A 2D graphics engine is a drawing engine for 2D drawings.

[0204] Android runtime is responsible for the scheduling and management of the Android system. Android runtime includes core libraries and virtual machines.

[0205] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0206] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0207] The kernel layer is the layer between hardware and software. The kernel layer can include display drivers, sensor drivers, microphone drivers, Wi-Fi drivers, etc.

[0208] Combination Fig. 9 , Fig.10 A schematic diagram showing the interaction of various modules of an electronic device when implementing the method for the voice control function provided in an embodiment of the present application is shown.

[0209] In the embodiment of the present application, the electronic device may pre-configure some hot words, or may obtain the text in the interface according to the display interface of the application running in the foreground, that is, obtain the hot words.

[0210] refer to Fig.10 In response to the electronic device starting Visible and Speakable, the screen element acquisition module acquires the elements of the display interface of the application running in the foreground of the electronic device (such as controls, text on controls, etc.). The screen element acquisition module transmits the acquired elements to the screen element parsing module, which parses the above elements to obtain the screen context (i.e., hot words). Then, the screen element parsing module transmits the hot words to the voice processing engine, which processes the user's voice and matches it with the hot words to generate corresponding instructions.

[0211] For example, in response to the electronic device starting Visible and Speakable, the electronic device collects the user's voice through the microphone and transmits the user's voice to the voice processing engine. The voice processing engine processes the user's voice and converts it into instructions that the electronic device finally executes. Among them, the voice processing engine includes voice recognition, intent understanding and dialogue management. Voice recognition converts voice signals into text information based on hot words; intent understanding converts text information into semantics that can be understood by electronic devices based on hot words, and understands the intent of the text information based on the semantics; dialogue management is used to generate instructions based on the understood intent of the text information.

[0212] After the voice processing engine generates the instruction, it transmits the instruction to the instruction distribution module. The instruction distribution module distributes the instruction to the multi-mode interaction module. The multi-mode interaction module obtains the package name of the application running in the foreground of the electronic device, and sends the instruction to the corresponding application, so that the application executes the instruction.

[0213] Combination Fig. 9 , Fig.11 A schematic diagram showing the interaction of various modules of an electronic device when implementing the refresh rate switching method provided in an embodiment of the present application is shown.

[0214] like Fig.11 As shown, in response to the electronic device starting to be visible and talkable, the registered monitoring module registers monitoring with the power module. In this way, when the power module monitors the power of the electronic device changes, it can call back to notify the registered monitoring module.

[0215] Correspondingly, the registered monitoring module registers monitoring with AMS. AMS can monitor the creation, modification, and destruction of the Activities of various applications on the electronic device. In this way, when AMS monitors the creation, modification, or destruction of an Activity, it can call back to notify the registered monitoring module. Among them, the notification includes the component name corresponding to the Activity, and the component name can include the application package name and the Activity.

[0216] For example, when application 1 is started, application 1 notifies AMS to create an Activity corresponding to the display interface of application 1. AMS callbacks the registered monitoring module to notify that the display interface of application 1 switches to the foreground, that is, application 1 runs in the foreground. Correspondingly, when application 1 exits and application 2 is started, application 2 notifies AMS to create an Activity corresponding to the display interface of application 2. AMS callbacks the registered monitoring module to notify that the display interface of application 2 switches to the foreground, that is, application 2 switches to the foreground and runs.

[0217] The registered monitoring module also registers monitoring with the sensor module. The sensor module can monitor the posture data of the electronic device, such as acceleration, angular velocity, distance from the user's face, etc. In this way, when the sensor module monitors the posture data of the electronic device, it can call back to notify the registered monitoring module.

[0218] Still Fig.11 As shown, after the registration monitoring module obtains the power and posture data of the electronic device, it transmits the power and posture data to the state management module; the state management module determines the state information of the electronic device according to the power and posture data of the electronic device. When the state information of the electronic device meets the preset conditions, the refresh rate switching module is notified.

[0219] When the AMS callback notifies the registered listening module and application 1 is running in the foreground, the registered listening module obtains the application name of application 1 and transmits the application name of application 1 to the scene perception module. The scene perception module determines the application type corresponding to application 1 based on the application name of application 1 and transmits the application type corresponding to application 1 to the refresh rate switching module.

[0220] Exemplarily, the scene perception module may include the correspondence between the application name and the application type preconfigured in the electronic device, for example, the scene perception module includes the correspondence shown in Table 4. The scene perception module may determine the application type corresponding to application 1 based on the correspondence shown in Table 4.

[0221] Continue to refer Fig.11 After the refresh rate switching module receives the notification that the status information of the electronic device meets the preset conditions, it determines the refresh rate B corresponding to the application type according to the application type transmitted by the scene perception module, and then transmits the refresh rate B to the refresh rate control module, which controls the display screen to display the display interface of application 1 at refresh rate B.

[0222] In one example, when AMS monitors that application 2 switches to the foreground, AMS calls back to notify the registered monitoring module, which obtains the application name of application 2 and transmits the application name of application 2 to the scene perception module. The scene perception module determines the application type corresponding to application 2 based on the application name of application 2, and transmits the application type corresponding to application 2 to the refresh rate switching module. The refresh rate switching module determines the corresponding refresh rate B based on the application type. The specific implementation method is similar to the above, and the above description can be referred to, which will not be repeated here.

[0223] In one example, when the status information of the electronic device does not meet the preset conditions, the registration monitoring module cancels the monitoring of the AMS, and there is no need to call back the notification registration monitoring module every time the application switches to the foreground, thereby reducing the power consumption generated by the need to call back the notification when the application switches.

[0224] In another example, when the electronic device exits visibility and speech, the registered monitoring module cancels monitoring of the power module and the sensor module, and there is no need to call back the volume and posture data of the electronic device to notify the registered monitoring module in real time, thereby reducing the power consumption generated by the callback notification.

[0225] In the embodiment of the present application, the electronic device may start the visible and audible mode, which may include: in response to the user's operation of a preset switch, the electronic device starts the visible and audible mode. Figure 2 The human-computer interaction interface shown turns on the "visible and talkable" function switch, making the electronic device visible and talkable.

[0226] Alternatively, when the electronic device is in a static state and the duration of the static state of the electronic device is greater than a preset duration (such as 3 seconds), the electronic device automatically starts to be visible and talkable.

[0227] Accordingly, the electronic device may exit being visible and talkable in a manner that: in response to the user's operation of a preset switch, the electronic device exits being visible and talkable. Figure 2 In the human-computer interaction interface shown, the "visible and talkable" function switch is turned off, so that the electronic device exits the visible and talkable state. Alternatively, when the electronic device switches from a static state to a moving state, the electronic device exits the visible and talkable state.

[0228] It should be noted that the above are only some examples of ways to start / exit visible and speakable functions on electronic devices. Currently, visible and speakable functions can also be started / exited through other feasible ways, which are subject to actual settings and are not listed here.

[0229] In addition, the contents recorded in each embodiment of the present application can explain the technical solutions in other embodiments of the embodiments of the present application. The technical features recorded in each embodiment can also be applied in other embodiments and combined with the technical features in other embodiments to form new solutions. The present application only exemplarily lists several embodiments for illustration, and does not mean that the present application is limited to this.

[0230] The present application provides an electronic device, which may include: a memory and one or more processors; the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the functions or steps executed in the above electronic device. The structure of the electronic device can refer to the above Figure 8 The structure of the electronic device 100 is shown.

[0231] The present application also provides a chip system, such as Fig.12 As shown, the chip system 1100 includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 may be Figure 8 The processor 110 is shown. The interface circuit 1102 may be, for example, an interface circuit between the processor 110 and an external memory; or an interface circuit between the processor and the internal memory 121 .

[0232] The processor 1101 and the interface circuit 1102 can be interconnected via lines. For example, the interface circuit 1102 can be used to receive signals from other devices (such as the memory of the electronic device 100). For another example, the interface circuit 1102 can be used to send signals to other devices (such as the processor 1101). Exemplarily, the interface circuit 1102 can read the instructions stored in the memory and send the instructions to the processor 1101. When the instructions are executed by the processor 1101, the electronic device can execute the various steps performed by the electronic device in the above embodiment. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiments of the present application.

[0233] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes each function or step executed by the electronic device in the above method embodiment.

[0234] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the electronic device in the above method embodiment.

[0235] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0236] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0237] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0238] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0239] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0240] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for switching a refresh rate, characterized in that: The invention is applied to an electronic device supporting a voice control function, wherein the voice control function allows a user to avoid saying a wake-up instruction, and after receiving a voice instruction input by the user, controls the electronic device to perform an operation corresponding to the voice instruction; the wake-up instruction is different from the voice instruction, and the wake-up instruction is used to wake up an application in the electronic device for voice interaction with the user; the method comprises: The electronic device starts the voice control function; In response to a voice input by a user to open a first application, the electronic device displays an interface of the first application at a first refresh rate; The electronic device detects that the state of the electronic device meets a preset condition, and displays the interface of the first application at a second refresh rate; The state of the electronic device satisfies a preset condition, which includes: the power of the electronic device is greater than a preset power, and the moving state of the electronic device is a stationary state; and the second refresh rate is less than the first refresh rate.

2. The method according to claim 1, characterized in that The second refresh rate is the minimum refresh rate allowed when the electronic device runs the first application in the foreground.

3. The method according to claim 1 or 2, characterized in that: The state of the electronic device satisfies a preset condition and further includes that the distance between the display screen of the electronic device and the user's face is greater than a preset value.

4. The method according to any one of claims 1 to 3, characterized in that The electronic device starts the voice control function, including: The electronic device displays a user interface for starting the voice control function, wherein the user interface includes a preset switch; in response to a user operating the preset switch, the electronic device starts the voice control function; or The moving state of the electronic device is a stationary state, and the duration of the electronic device being in the stationary state is greater than a preset duration, and the electronic device activates the voice control function.

5. The method according to any one of claims 1 to 4, characterized in that The displaying the interface of the first application at the second refresh rate includes: The electronic device obtains an application name of the first application; The electronic device determines, according to the application name of the first application, an application type corresponding to the first application; The electronic device displays the interface of the first application at the second refresh rate according to the application type corresponding to the first application.

6. The method according to claim 5, characterized in that The electronic device displays the interface of the first application at the second refresh rate according to the application type corresponding to the first application, including: If the application type corresponding to the first application is an audio and video application, and the first refresh rate is greater than the third refresh rate, the electronic device uses the second refresh rate to display the interface of the first application; wherein the third refresh rate is less than the first refresh rate and greater than the second refresh rate; or, If the application type corresponding to the first application is an e-book application, the electronic device displays the interface of the first application using a second refresh rate.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: The electronic device exits the voice control function, and the electronic device displays the interface of the first application at the first refresh rate; or, The state of the electronic device does not satisfy a preset condition, and the electronic device displays the interface of the first application using the first refresh rate.

8. The method according to any one of claims 1 to 7, characterized in that The electronic device includes a voice control application package APK, and the voice control APK is used to control the electronic device to perform the voice control function; after the electronic device starts the voice control function, the method further includes: The electronic device registers for monitoring with the application framework layer through the voice control APK of the application layer; When the electronic device opens the first application, the application framework layer of the electronic device calls back to notify the voice control APK of the application layer; The callback notification at least includes the state of the electronic device and the application name of the first application.

9. The method according to claim 8, characterized in that The method further comprises: When the state of the electronic device does not meet the preset condition, the electronic device cancels the registration monitoring with the application framework layer through the voice control APK of the application layer; or, When the electronic device exits the voice control function, the electronic device cancels the registration monitoring with the application framework layer through the voice control APK of the application layer.

10. An electronic device, characterized in that: The electronic device supports a voice control function, wherein the voice control function allows the user to avoid saying a wake-up instruction, and after receiving a voice instruction input by the user, controls the electronic device to perform an operation corresponding to the voice instruction; the wake-up instruction is different from the voice instruction, and the wake-up instruction is used to wake up an application in the electronic device for voice interaction with the user; the electronic device comprises: memory and one or more processors; The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method as claimed in any one of claims 1 to 9.

11. A chip system, characterized in that: Applicable to an electronic device supporting a voice control function, wherein the voice control function allows the user to avoid saying a wake-up instruction, and after receiving a voice instruction input by the user, controls the electronic device to perform an operation corresponding to the voice instruction; the wake-up instruction is different from the voice instruction, and the wake-up instruction is used to wake up an application in the electronic device for voice interaction with the user; the chip system includes: at least one processor and an interface; The interface is used to receive instructions and transmit them to the at least one processor; the at least one processor executes the instructions so that the electronic device executes the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method as claimed in any one of claims 1 to 9.

Images