Control method of electronic equipment, electronic equipment, storage medium and chip

Abstract

The invention provides a control method of electronic equipment, the electronic equipment, a storage medium and a chip. According to the method, the voice control video function of the electronic equipment can be started, and after the function is started, when the electronic equipment plays a video, a user can input a voice instruction to the electronic equipment in a voice mode under the condition that the mode that the user inputs the instruction to the electronic equipment through operations such as clicking and touching is limited; the electronic equipment can control the played video according to the voice instruction of the user, so that the user experience is improved. According to the method, under the condition that the user operation is not detected within the preset duration, the user is inconvenient to input the instruction to the electronic equipment through the user operation, and the electronic equipment can control the played video based on the automatic opening of the voice control video function according to the voice instruction of the user, so that the user experience is improved.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 3, 2023, with application number 202311462542.6 and application name “A scene-perception-based intelligent assistance method and electronic device”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of terminals, and in particular to a control method of an electronic device, an electronic device, a storage medium and a chip. Background Art

[0003] Currently, electronic devices such as mobile phones and tablet computers can provide many functions. For example, mobile phones can provide the function of playing videos and providing the function of multi-person video conferencing. Users can interact with electronic devices and experience the functions provided by the electronic devices.

[0004] However, in the related solutions, the interaction mode between the user and the electronic device is relatively simple. When the user uses the electronic device with a relatively simple interaction mode, for example, when the user uses the electronic device with a relatively simple interaction mode to watch the video played by the electronic device, the user can input instructions to the electronic device through operations such as clicking and touching to control the video played by the electronic device. For example, the user can adjust the video played by the electronic device by clicking the volume control of the electronic device, and the user can touch the display screen of the electronic device to pause the video being played by the electronic device, etc. The above-mentioned method of inputting instructions to the electronic device may be limited. For example, when the electronic device is playing a video, the user is handling other matters (such as cooking, sweeping the floor, etc.), and it is inconvenient for the user to input instructions to the electronic device through operations such as clicking and touching, which affects the user experience. Summary of the invention

[0005] The present application provides a control method of an electronic device, an electronic device, a storage medium and a chip, which can input commands to the electronic device through another input command method when a user is restricted from inputting commands to the electronic device through one input command method, thereby improving the user experience.

[0006] In a first aspect, a control method for an electronic device is provided, the method comprising: displaying a first interface, the first interface including a first control; in response to an operation on the first control, turning on a voice control video function of the electronic device; displaying a second interface, the second interface displaying a video; controlling the video displayed on the second interface according to a voice instruction; or; displaying a second interface; detecting a user operation on the electronic device; and controlling the video displayed on the second interface based on automatically turning on the voice control video function according to a voice instruction when no user operation is detected within a preset time.

[0007] In a related scheme, when a user uses an electronic device with a relatively simple interaction method to watch a video played by the electronic device, the user can input instructions to the electronic device through operations such as clicking and touching to control the video played by the electronic device. For example, the user can adjust the video played by the electronic device by clicking the volume control of the electronic device, and the user can touch the display screen of the electronic device to pause the video being played by the electronic device. However, the above-mentioned method of inputting instructions to the electronic device may be restricted. For example, while the electronic device is playing a video, the user is handling other matters (such as cooking, sweeping the floor, etc.), it is inconvenient for the user to input instructions to the electronic device through operations such as clicking and touching, which affects the user experience.

[0008] In an embodiment of the present application, the electronic device can enable the voice control video function of the electronic device through the first control included in the first interface. After the function is enabled, when the electronic device displays the second interface (the electronic device plays a video on the second interface), when the user is restricted from inputting commands to the electronic device through operations such as clicking and touching, the user can input voice commands to the electronic device in the form of voice, and the electronic device can control the played video according to the user's voice commands, thereby improving the user experience.

[0009] In an embodiment of the present application, when the electronic device displays the second interface (the electronic device plays a video on the second interface), it can detect the user's operation on the electronic device (the user inputs instructions to the electronic device through user operations (such as clicking, touching, etc.)). If no user operation is detected within a preset time period (this situation proves that the way in which the user inputs instructions to the electronic device through user operation is limited), and it is inconvenient for the user to input instructions to the electronic device through user operation, the electronic device can control the played video based on the user's voice command and automatically start the voice control video function (the automatic voice control video function can be turned on before detecting the user's operation on the electronic device, and can also be turned on when no user operation is detected within a preset time period), thereby improving the user experience.

[0010] In combination with the first aspect, in certain implementations of the first aspect, before displaying the first interface, the method also includes: displaying a third interface, the third interface including a first icon; in response to an operation on the first icon, displaying a fourth interface, the fourth interface including a second control; in response to an operation on the second control, displaying a fifth interface, the fifth interface including a third control; in response to an operation on the third control, displaying a sixth interface, the sixth interface including a fourth control; and displaying the first interface, including: displaying the first interface in response to an operation on the fourth control.

[0011] In an embodiment of the present application, the electronic device can display a first interface in response to a user's operation on a first icon, a second control, a third control, or a fourth control, and can enable a voice control video function in response to a user's operation on a first control in the first interface. Through the above operations, the user can quickly and conveniently enable the voice control video function of the electronic device, thereby improving the user experience.

[0012] In combination with the first aspect, in some implementations of the first aspect, before displaying the first interface, the method further includes:

[0013] Displaying a fourth interface, the fourth interface including a fifth control; in response to an operation on the fifth control, displaying a seventh interface, the seventh interface including a sixth control; and displaying the first interface, including: displaying the first interface in response to an operation on the sixth control.

[0014] An embodiment of the present application provides another way for a user to turn on a control video function. Specifically, the electronic device can display a first interface in response to the user's operation on a fifth control and an operation on a sixth control, and can turn on a voice control video function in response to the user's operation on a first control in the first interface. The user can turn on the voice control video function of the electronic device through the above operations. The electronic device provides the user with a variety of ways to turn on the voice control video function of the electronic device, thereby improving the user experience.

[0015] In combination with the first aspect, in some implementations of the first aspect, the second interface includes a first capsule; and before detecting a user operation on the electronic device, the method further includes:

[0016] In response to the operation on the first capsule, a first pop-up window is displayed, and the first pop-up window includes a seventh control; in response to the operation on the seventh control, the automatic voice control video function is turned on.

[0017] In an embodiment of the present application, before detecting a user operation on the electronic device, the electronic device can respond to the user's operation on the first capsule and the operation on the seventh control in the first pop-up window to turn on the automatic voice control video function. Through the above operations, the user can quickly and conveniently enable the automatic voice control video function of the electronic device to improve the user experience.

[0018] In combination with the first aspect, in certain implementations of the first aspect, when no user operation is detected within a preset time period, the video displayed on the second interface is controlled based on the automatic start of the voice control video function according to the voice command, including: when no user operation is detected within the preset time period, a first pop-up window is displayed, the first pop-up window includes a seventh control; in response to the operation of the seventh control, the automatic start of the control video function is turned on; according to the voice command, the video displayed on the second interface is controlled.

[0019] In an embodiment of the present application, the electronic device can prompt the user through a first pop-up window to enable the automatic start of voice control video function when no user operation is detected within a preset time period. Specifically, the electronic device can prompt the user to enable the automatic start of voice control video function through the seventh control in the first pop-up window. The user can enable the automatic start of voice control video function according to the above prompt, thereby improving the user experience.

[0020] In a second aspect, a control method for an electronic device is provided, the method comprising: displaying an eighth interface, the eighth interface including an eighth control; in response to an operation on the eighth control, turning on a subtitle function of the electronic device; or: displaying a third interface; in response to an operation on the third interface, displaying a ninth interface, the ninth interface including a ninth control; in response to an operation on the ninth control, turning on the subtitle function of the electronic device; displaying a second interface, the second interface displaying a video; displaying subtitles corresponding to the voice of the video in the second interface; or; displaying the second interface, the second interface displaying a video; detecting a video file corresponding to the video: when it is detected that the video file does not include subtitle information, displaying subtitles corresponding to the voice of the video in the second interface based on automatically turning on the subtitle function.

[0021] In a related solution, when a user uses a video function of an electronic device with a relatively simple interaction method (for example, the user uses the electronic device to conduct a video conference), the electronic device can provide feedback to the user on the content of the video conference by playing the voice of the video. However, the manner in which the above-mentioned electronic device provides feedback to the user may be limited. For example, if the noise intensity of the surrounding environment of the electronic device is too high, or the volume of the voice of the video played by the electronic device is too low or muted, the user cannot hear the voice of the video clearly, which affects the user experience.

[0022] In an embodiment of the present application, the electronic device can enable the subtitle function of the electronic device through the eighth control in the eighth interface, or through the third interface and the ninth control in the ninth interface. After the function is turned on, when the user uses the video function of the electronic device (the second interface displays a video, and the user conducts a video conference through the video displayed on the second interface), when the electronic device is limited in the way of providing feedback to the user through the voice of the video being played, the electronic device can provide feedback to the user by displaying subtitles corresponding to the voice of the video in the second interface, thereby improving the user experience.

[0023] In an embodiment of the present application, when displaying the second interface, the electronic device can detect the video file corresponding to the video. When it is detected that the video file does not include subtitle information (this situation proves that the way in which the electronic device feeds back information to the user through the voice of the video is limited), the electronic device can display the subtitles corresponding to the voice of the video in the second interface based on the automatic subtitle activation function (the automatic subtitle activation function can be activated before detecting the video file corresponding to the video, and can also be activated when it is detected that the video file does not include subtitle information), thereby improving the user experience.

[0024] In combination with the second aspect, in certain implementations of the second aspect, before displaying the eighth interface, the method also includes: displaying a third interface, the third interface including a first icon; in response to an operation on the first icon, displaying a fourth interface, the fourth interface including a second control; in response to an operation on the second control, displaying a fifth interface, the fifth interface including a third control; in response to an operation on the third control, displaying a sixth interface, the sixth interface including a tenth control; and displaying the eighth interface, including: displaying the eighth interface in response to an operation on the tenth control.

[0025] In the embodiment of the present application, the electronic device can display the eighth interface in response to the operation of the first icon, the operation of the second control, the operation of the third control, and the operation of the tenth control, and can enable the subtitle function in response to the user's operation of the eighth control in the eighth interface. The user can use the above operations to quickly and conveniently enable the subtitle function of the electronic device, thereby improving the user experience.

[0026] In combination with the second aspect, in some implementations of the second aspect, the second interface includes a second capsule; and before detecting the video file corresponding to the video, the method also includes: in response to an operation on the second capsule, displaying a second pop-up window, the second pop-up window including an eleventh control; in response to the operation on the eleventh control, turning on the automatic subtitle opening function.

[0027] In an embodiment of the present application, before the electronic device detects the video file corresponding to the video, the electronic device can respond to the operation of the second capsule and the operation of the eleventh control in the second pop-up window to turn on the automatic subtitle opening function. Through the above operations, the user can quickly and conveniently enable the automatic subtitle opening function of the electronic device to improve the user experience.

[0028] In combination with the second aspect, in certain implementations of the second aspect, when it is detected that the video file does not include subtitle information, based on the automatic subtitle opening function, subtitles corresponding to the voice of the video are displayed in the second interface, including: when it is detected that the video file does not include subtitle information, a second pop-up window is displayed, and the second pop-up window includes an eleventh control; in response to an operation on the eleventh control, the automatic subtitle opening function is turned on; and subtitles corresponding to the voice of the video are displayed in the second interface.

[0029] In an embodiment of the present application, the electronic device can prompt the user through a second pop-up window to enable the automatic subtitle opening function when it detects that the video file does not include subtitle information. Specifically, the electronic device can prompt the user to enable the automatic subtitle opening function through an eleventh control in the second pop-up window. The user can enable the automatic subtitle opening function according to the above prompt, thereby improving the user experience.

[0030] In combination with the second aspect, in certain implementations of the second aspect, detecting a video file corresponding to a video includes: detecting subtitle information in the video file; obtaining a volume value when the electronic device plays the video when it is detected that the video file does not include subtitle information; displaying subtitles corresponding to the voice of the video in a second interface based on an automatic subtitle function when it is detected that the video file does not include subtitle information, including: displaying subtitles corresponding to the voice of the video in a second interface based on an automatic subtitle function when it is determined that the volume value when the electronic device plays the video is less than a preset volume value.

[0031] In an embodiment of the present application, the electronic device can quickly and accurately locate the scenario where the way in which the electronic device feeds back information to the user through the voice of the video is restricted by detecting that the video file does not include subtitle information and determining that the volume value of the electronic device when playing the video is less than a preset volume value. After locating the above scenario, the electronic device can automatically turn on the subtitle function and feed back information to the user by displaying subtitles corresponding to the voice of the video in the second interface, instead of feeding back information to the user by displaying subtitles corresponding to the voice of the video during the entire process of the user using the video function of the electronic device, thereby reducing the time the electronic device displays subtitles and reducing the power consumption of the electronic device.

[0032] In combination with the second aspect, in certain implementations of the second aspect, detecting a video file corresponding to a video includes: detecting subtitle information in the video file; when it is detected that the video file does not include subtitle information, obtaining a noise value of a current environment in which the electronic device is located; when it is detected that the video file does not include subtitle information, based on automatically starting a subtitle function, displaying subtitles corresponding to the voice of the video in a second interface, including: when it is determined that the noise value of the current environment in which the electronic device is located is less than a preset noise value, based on automatically starting a subtitle function, displaying subtitles corresponding to the voice of the video in a second interface.

[0033] In an embodiment of the present application, the electronic device can quickly and accurately locate the scenario where the electronic device is restricted from providing feedback to the user through the voice of the video being played by detecting that the video file does not include subtitle information and determining that the noise value of the current environment in which the electronic device is located is less than a preset noise value. After locating the above scenario, the electronic device can provide feedback to the user by displaying subtitles corresponding to the voice of the video in a second interface based on automatically turning on the subtitle function, without providing feedback to the user by displaying subtitles corresponding to the voice of the video during the entire process of the user using the video function of the electronic device, thereby reducing the time the electronic device displays subtitles and reducing the power consumption of the electronic device.

[0034] In combination with the second aspect, in certain implementations of the second aspect, detecting a video file corresponding to a video includes: detecting subtitle information in the video file; when it is detected that the video file does not include subtitle information, obtaining an audio language corresponding to the video; and when it is detected that the video file does not include subtitle information, based on automatically starting a subtitle function, displaying subtitles corresponding to the voice of the video in a second interface, including: when it is determined that the audio language corresponding to the video is different from the system language of the electronic device, based on automatically starting a subtitle function, displaying subtitles corresponding to the voice of the video in the second interface.

[0035] In an embodiment of the present application, the electronic device can quickly and accurately locate the scenario where the way in which the electronic device feeds back information to the user through the voice of the video is restricted by detecting that the video file does not include subtitle information and determining that the audio language corresponding to the video is different from the system language of the electronic device. After locating the above scenario, the electronic device can automatically turn on the subtitle function and feed back information to the user by displaying the subtitles corresponding to the voice of the video in the second interface, instead of feeding back information to the user by displaying the subtitles corresponding to the voice of the video during the entire process of the user using the video function of the electronic device, thereby reducing the time the electronic device displays subtitles and reducing the power consumption of the electronic device.

[0036] In a third aspect, an electronic device is provided, comprising: one or more processors; one or more memories; the one or more memories storing one or more computer programs, the one or more computer programs comprising instructions, which, when executed by the one or more processors, enable the electronic device to execute a method in any possible implementation of the first aspect.

[0037] In a fourth aspect, a computer-readable storage medium is provided, comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method described in the first aspect.

[0038] In a fifth aspect, a chip is provided, comprising a memory for storing instructions; and a processor for calling and executing instructions from the memory, so that an electronic device equipped with the chip executes the method described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a structural diagram of an electronic device 100 provided in an embodiment of the present application.

[0040] Figure 2 It is a schematic diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0041] Figure 3 A schematic diagram of a user interacting with an electronic device is provided.

[0042] Figure 4 A schematic diagram is provided when the interaction method between a user and an electronic device is restricted.

[0043] Figure 5 This is a schematic diagram of a user interacting with an electronic device provided in an embodiment of the present application.

[0044] Figure 6 It is a schematic diagram of enabling a voice control video function provided in an embodiment of the present application.

[0045] Figure 7 This is another schematic diagram of enabling the voice control video function provided in an embodiment of the present application.

[0046] Figure 8 This is another schematic diagram of enabling the voice control video function provided in an embodiment of the present application.

[0047] Fig. 9 This is another schematic diagram of enabling the voice control video function provided in an embodiment of the present application.

[0048] Fig.10 This is an example diagram of turning off the automatic voice control video function provided in an embodiment of the present application.

[0049] Fig.11 This is another schematic diagram of user interaction with an electronic device provided in an embodiment of the present application.

[0050] Fig.12 This is an example diagram of an interface 1210 provided in an embodiment of the present application.

[0051] Fig.13 This is a schematic diagram of a function for enabling AI subtitles provided in an embodiment of the present application.

[0052] Fig.14 This is a schematic diagram of another function of enabling AI subtitles provided in an embodiment of the present application.

[0053] Fig.15 This is a schematic diagram of another function of enabling AI subtitles provided in an embodiment of the present application.

[0054] Fig.16 This is a schematic diagram of an embodiment of the present application that prompts a user to enable a function of automatically turning on AI subtitles.

[0055] Fig.17 This is an example diagram of a function of turning off AI subtitles provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" or "multiple" refers to two or more than two.

[0057] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0058] The control method of the electronic device provided in the embodiment of the present application is applied to the electronic device. The electronic device may also be referred to as a terminal, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The electronic device may be an electronic device (mobile phone), a smart TV, a wearable device, a tablet computer (Pad), a computer with a wireless transceiver function, a virtual reality (VR) electronic device, an augmented reality (AR) electronic device, a wireless terminal in industrial control (industrial control), a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid (smart grid), a wireless terminal in transportation safety (transportation safety), a wireless terminal in a smart city (smart city), a wireless terminal in a smart home (smart home), etc. The embodiment of the present application does not limit the specific technology and specific device form adopted by the electronic device.

[0059] Figure 1 1 is a schematic diagram of the structure of an electronic device 100 (taking a mobile phone as an example) provided in an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, an antenna 1, an antenna 2, a mobile communication module 140, a wireless communication module 150, a sensor module 160, a pressure sensor 160A, a touch sensor 160B, a display screen 170, etc.

[0060] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0061] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0062] Specifically in the embodiments of the present application, the processor 110 can determine whether the application is running, determine whether a user operation is received within a preset time period, determine whether the video content has human voice audio, determine whether the video content presented by the video application has subtitle information, determine whether the volume value of the electronic device is less than a preset volume value, etc.

[0063] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0064] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0065] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0066] For example, the USB interface 130 is an interface that complies with USB standard specifications, and specifically can be a Mini USB interface, a MicroUSB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones.

[0067] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 140, the wireless communication module 150, the modem processor and the baseband processor.

[0068] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0069] The mobile communication module 140 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 140 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 140 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 140 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 140 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 140 can be set in the same device as at least some of the modules of the processor 110.

[0070] The wireless communication module 150 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 150 can be one or more devices integrating at least one communication processing module. The wireless communication module 150 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 150 can also receive the signal to be sent from the processor 110, modulate the frequency of the signal, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0071] The display screen 170 is used to display images, videos, etc.

[0072] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0073] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0074] The pressure sensor 160A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 160A can be disposed on the display screen 170. There are many types of pressure sensors 160A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc.

[0075] Specifically in the embodiment of the present application, the electronic device 100 can detect user operations through the pressure sensor 160A.

[0076] The touch sensor 160B is also called a "touch panel". The touch sensor 160B can be disposed on the display screen 170. The touch sensor 160B and the display screen 170 form a touch screen, also called a "touch screen". The touch sensor 160B is used to detect a touch operation on or near it.

[0077] Specifically in the embodiment of the present application, the electronic device 100 can also detect user operations through the touch sensor 160B.

[0078] The hardware structure of the electronic device 100 is introduced here. It can be understood that: Figure 1 The components included in the hardware structure shown do not constitute a specific limitation on the electronic device 100. The electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0079] In addition, an operating system is running on the above components, such as the iOS operating system developed by Apple, the Android open source operating system developed by Google, and the Windows operating system developed by Microsoft. Application programs can be installed and run on the operating system.

[0080] The operating system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the operating system of the electronic device 100.

[0081] Figure 2 1 is a schematic diagram of the software structure of the electronic device 100 of the embodiment of the present application. The software structure includes several layers, each layer has a clear role and division of labor, and the layers communicate with each other through software interfaces. In some embodiments, the software structure is divided into an application layer, an application framework layer, a system library, and a kernel layer from top to bottom.

[0082] The application layer can include a series of application packages, such as camera, gallery, calendar, map, navigation, etc.

[0083] The application framework layer, referred to as the framework layer, provides APIs and programming frameworks for the applications in the application layer. The framework layer includes some predefined functions. When the application in the application layer is run, the relevant functions of the application can be implemented by calling the API. The framework layer may include a window manager, a phone manager, a resource provider, and a content provider, etc.

[0084] System libraries can include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0085] The kernel layer can include multiple driver modules, such as display driver, camera driver, audio driver, and sensor driver.

[0086] Understandably, Figure 2 The layers in the software structure shown and the components contained in each layer do not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer layers than shown in the figure, and each layer may include more or fewer components. For example, the software structure may also include a hardware abstraction layer, a hardware layer, etc., which is not limited in the present application.

[0087] It is understandable that, in order to implement the control method of the electronic device in the embodiment of the present application, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments.

[0088] It should be noted that although the embodiments of the present application are described using the Android system as an example, its basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0089] The executor of the control method of the electronic device provided in the embodiment of the present application can be the above-mentioned electronic device, or it can be a functional module and / or functional entity in the electronic device that can implement the control method of the electronic device, and the present application scheme can be implemented by hardware and / or software, which can be determined according to actual usage requirements and is not limited by the embodiment of the present application.

[0090] The technical problems to be solved by this application and the inventive concept of this application are introduced below.

[0091] At present, electronic devices such as mobile phones and tablet computers can provide many functions, such as the function of playing videos and the function of multi-person video conferencing. Users can interact with electronic devices and experience the functions provided by electronic devices. For example, please refer to Figure 3 , Figure 3 A schematic diagram of a user interacting with an electronic device is provided. Figure 3 (a) shows that a user inputs instructions to an electronic device by clicking or touching the electronic device, using sound, gestures, eye movements, body movements, etc., and the electronic device receives the user's input instructions through input devices such as a display screen, microphone, camera, and sensor. Figure 3 (b) shows that the electronic device provides feedback information to the user through images, videos, audio, vibration, etc. according to the input instructions, and the user receives the feedback information from the electronic device through hearing, vision, touch, etc., thereby realizing information exchange and interaction between the user and the electronic device, allowing the user to experience the functions provided by the electronic device.

[0092] However, in the related solutions, the interaction mode between the user and the electronic device is relatively simple. The relatively simple interaction mode may be that the user inputs a command to the electronic device in a relatively simple manner, for example, when the user is watching a video through the electronic device, the user inputs a command to pause the video to the electronic device by clicking a pause control provided by the electronic device. The relatively simple interaction mode may also be that the electronic device provides feedback information to the user in a relatively simple manner, for example, when the user is having a video conference through the electronic device, the electronic device provides feedback information to the user by playing audio to enable the user to understand the content of the conference.

[0093] When users use electronic devices with a relatively simple interaction method, the way users input commands or the way electronic devices feedback information may be limited, affecting the user experience. For example, please refer to Figure 4 , Figure 4 Provided is a schematic diagram of when the user's interaction with an electronic device is restricted. Figure 4 (a) shows that the mobile phone is playing a video while the user is cooking. When the user needs to pause the video being played, the user originally needs to input a command to the mobile phone by clicking the pause control provided by the mobile phone (input a command to pause the video). However, the above method of inputting commands is limited because the user is cooking. It is inconvenient for the user to pause the video by the above method of inputting commands, which affects the user experience. Figure 4 (b) shows that when a user is conducting a video conference via a mobile phone on the subway, the user's surrounding environment is relatively noisy. The electronic device originally feeds back information to the user (feedback on the meeting content) by playing audio, but the above feedback information method is limited due to the noisy surrounding environment. The user cannot hear the meeting content fed back by the electronic device through the above feedback information method, affecting the user experience. Figure 4 (c) shows that during the riding process, the user originally inputs instructions to the mobile phone (inputs instructions to answer the call) through the answering call controls provided by the mobile phone, but the above method of inputting instructions is restricted because the user is riding, and it is inconvenient for the user to answer the call through the above method of inputting instructions, which affects the user experience.

[0094] In order to solve the above problems, an embodiment of the present application provides a method for controlling an electronic device, which can input instructions to the electronic device in another way when the user is restricted from inputting instructions to the electronic device in one way. For example, when the user is restricted from inputting instructions to the mobile phone by clicking the controls provided by the mobile phone to control the video, the user can input instructions to the mobile phone by voice. The method can also feedback information to the user in another way when the electronic device is restricted from feeding back information to the user in one way. For example, when the electronic device is restricted from feeding back information to the user through audio, the user can be fed back through subtitles, thereby improving the user experience.

[0095] The following takes the electronic device being a mobile phone as an example, and describes the control method of the electronic device provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0096] Before introducing the specific implementation examples in detail, the term "user interface (UI)" used in the application scenario is first introduced. UI is a medium interface for interaction and information exchange between an application or operating system and a user. It realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the electronic device, and finally presented as content that the user can recognize, such as pictures, text, buttons and other controls. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, pictures and text. The properties and contents of controls in the interface are defined by tags or nodes, such as XML through, <textview> 、 <imgview> 、 <videoview>The nodes are used to define the controls contained in the interface. A node corresponds to a control or attribute in the interface. After parsing and rendering, the node is presented as user-visible content.

[0097] The most common form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be an icon, window, control or other interface element displayed on the display screen of an electronic device, where a control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements.

[0098] The following embodiment introduces an application scenario of the embodiment of the present application in conjunction with the accompanying drawings. The application scenario is: when the method in which a user inputs a command to an electronic device is restricted, the user can input a command to the electronic device through another input command method.

[0099] Please refer to Figure 5 , Figure 5 It is a schematic diagram of a user interacting with an electronic device provided in an embodiment of the present application. It illustrates that: while a user is watching a video through an electronic device, he is performing other tasks (such as cooking). The user originally needs to input video control instructions to the mobile phone by operating the control controls provided by the mobile phone (such as pause controls, play controls, and fast forward controls, etc.), but the above-mentioned method of inputting video control instructions is limited because the user is performing other tasks. The user can input video control instructions to the mobile phone through other input instruction methods. Other methods of inputting video control instructions can be sound, gestures, eye movements, body movements, etc. The embodiment of the present application takes the other method of inputting video control instructions as sound as an example for explanation.

[0100] For example, please refer to Figure 5 In (a), the user can input the instruction of "fast forward 10 seconds" to the electronic device through voice, and the electronic device responds to the user's instruction of "fast forward 10 seconds" by displaying icon 510 and controlling the video to fast forward 10 seconds. Figure 5 In (b), the user can input a "pause" command to the electronic device through voice, and the electronic device responds to the user's "pause" command by displaying icon 520 and controlling the video to pause. Figure 5 In (b), the user can input a "pause" command to the electronic device through voice, and the electronic device responds to the user's "pause" command by displaying icon 520 and controlling the video to pause. Figure 5 In (c), the user can input a "play" command to the electronic device through voice, and the electronic device responds to the user's "play" command, displays icon 530 and controls the video playback. In other embodiments, the command input by the user to the electronic device through voice can also be: "fast rewind 10 seconds", "play next episode", etc., which is not limited in the embodiments of the present application.

[0101] In the embodiments of the present application, when the way a user inputs commands to an electronic device is restricted (for example, the way a user inputs commands to a mobile phone by operating the mobile phone is restricted), the user can input commands to the electronic device by other ways of inputting commands (such as voice, gesture, etc.), which can improve the user experience compared to related solutions in which the way a user inputs commands to an electronic device is relatively single.

[0102] It should be understood that users Figure 5 Before inputting instructions to an electronic device in the manner shown, the voice control video function of the electronic device needs to be turned on. The following is an introduction to the manner in which the voice control video function of the electronic device is turned on.

[0103] Please refer to Figure 6 , Figure 6 1 is a schematic diagram of enabling a voice control video function provided by an embodiment of the present application, which illustrates a process in which a user manually enables a voice control video function in an electronic device.

[0104] Figure 6 The interface 610 shown in (a) includes an icon 611, and the user can click the icon 611. In response to the user clicking the icon 611, the electronic device displays Figure 6 The interface 620 shown in (b) of FIG. The interface 620 includes a control 621, and the user can click the control 621. In response to the user clicking the control 621, the electronic device displays Figure 6 The interface 630 shown in (c) of FIG. The interface 630 includes a control 631, and the user can click the control 631. In response to the user clicking the control 631, the electronic device displays Figure 6 The interface 640 shown in (d) in FIG. The interface 640 includes a control 641, and the user can click the control 641. In response to the user clicking the control 641, the electronic device displays Figure 6 The interface 650 shown in (e1) in FIG. The interface 650 includes a control 651, Figure 6 The control 651 in (e1) is in the closed state, and the user can click or slide the control 651. In response to the user's operation of clicking or sliding the control 651, the electronic device can control the control 651 to switch from the closed state to the open state, for example, Figure 6 The control 651 shown in (e2) is in the turned-on state, and the control 651 refers to the control for turning on or off the voice control video function.

[0105] Please refer to Figure 7 , Figure 7 1 is another schematic diagram of enabling the voice control video function provided by an embodiment of the present application, which illustrates another process of a user manually enabling the voice control video function in an electronic device.

[0106] Figure 6 The interface 620 shown in (b) also includes a control 622, and the user can click the control 622. In response to the user clicking the control 622, the electronic device displays Figure 7 The interface 710 shown in (a1) in FIG. The interface 701 includes a control 711, Figure 7 The control 711 shown in (a1) is in a closed state. The user can click or slide the control 711. In response to the user clicking or sliding the control 711, the electronic device controls the control 711 to switch from the closed state to the open state. Figure 7 (a2) in FIG. 7 shows that the control 711 is in the on state. The control 711 is a control for turning on or off the voice control video function.

[0107] By the above Figure 6 and Figure 7 It can be seen that the electronic device in the embodiment of the present application provides a variety of controls for turning on the voice control video function. The user can turn on the voice control video function in a variety of ways. When the user has not mastered or forgotten one way to turn on the voice control video function, the user can turn on the voice control video function in another way, thereby improving the user experience.

[0108] Please refer to Figure 8 , Figure 8 1 is another schematic diagram of enabling a voice control video function provided by an embodiment of the present application, which illustrates: a process of enabling an electronic device to automatically enable a voice control video function.

[0109] Users can click Figure 6 In the control 641 shown in (d), in response to the user clicking the control 641, the electronic device displays Figure 8 The interface 810 shown in (a1) in FIG. The interface 810 includes a control 811, Figure 8 The control 811 shown in (a1) is in a closed state, and the user can click or slide the control 811. In response to the user's operation of clicking or sliding the control 811, the electronic device can control the control 811 to switch from the closed state to the open state. Figure 8 The control 811 shown in (a2) is in the turned-on state.

[0110] After the control 811 is turned on, when the user is watching a video through the electronic device, when the electronic device determines that the way in which the user inputs instructions to the electronic device is restricted (for example, when the electronic device determines that the way in which the user inputs instructions to the electronic device by operating the control controls of the electronic device is restricted), the electronic device can enable the automatic voice control video function. After the automatic voice control video function is enabled, the user can Figure 5 The method shown is to input instructions to the electronic device.

[0111] In the embodiment of the present application, the implementation process of the electronic device determining whether the way in which the user inputs an instruction to the electronic device is restricted may be:

[0112] In step A, the electronic device determines whether an entertainment application is running.

[0113] It should be understood that entertainment applications refer to apps that provide users with various entertainment and leisure methods. For example, users can watch videos, read e-books, and read news and information through entertainment apps.

[0114] In implementation, the electronic device can determine whether the entertainment application is running by detecting whether the entertainment app is running in the foreground. For example, the electronic device obtains the process list of the currently running application, queries the application process list, and assumes that the package name of the entertainment application is queried in the application process list, then it is determined that the entertainment app is running in the foreground. Assuming that the package name of the entertainment application is not queried in the application process list, then it is determined that the entertainment app is not running in the foreground. The electronic device can also determine whether the entertainment application is running by monitoring the life cycle of the entertainment application. For example, the electronic device registers a listener to monitor the life cycle of the entertainment application, such as the start, run, pause, stop and destruction of the entertainment application. Assuming that the listener monitors that the entertainment application is running, the electronic device determines that the entertainment application is running. The embodiment of the present application defines the implementation steps for determining whether the entertainment application is running.

[0115] Step B: When the electronic device determines that an entertainment application is running, it determines whether a user operation is received within a preset time period.

[0116] It should be understood that the preset duration may be 5 minutes, 10 minutes, etc., and this embodiment of the present application is not limited to this.

[0117] In implementation, the electronic device can detect whether the user operates the electronic device through sensors (such as touch sensors, pressure sensors, etc.) in the electronic device. When the sensor detects the user operation, the sensor reports the detected user operation information to the processor in the electronic device. When the sensor does not detect the user operation, it will not report the user operation information to the processor. Assuming that the processor does not receive the user operation information reported by the sensor within the preset time length, it is determined that no user operation is received within the preset time length.

[0118] Step C: when it is determined that no user operation is received within a preset time period, it is determined that the method for the user to input instructions to the electronic device is restricted.

[0119] It should be understood that if the electronic device does not receive any user operation within the preset time period, it means that the user is restricted in the way he or she operates the control controls of the electronic device or entertainment applications to input commands to the electronic device. For example, if a user is cooking while watching a video on the electronic device, since the user is cooking, the user cannot operate the video App to input operations such as pausing the video, playing the video, fast forwarding, etc., resulting in the user's way of inputting commands to the electronic device being restricted.

[0120] In the embodiment of the present application, the electronic device can enable the automatic voice control video function when it is precisely located that the user is restricted from inputting instructions to the electronic device by operating the control components of the electronic device, and through the above Figure 6 and Figure 7 After manually turning on the voice control video function, the user can control the video by voice no matter whether the way the user operates the control controls of the electronic device is restricted or not. In other words, the program corresponding to the voice control video is running no matter whether the way the user inputs information to the electronic device is restricted or not, which increases the power consumption of the electronic device. The embodiment of the present application can accurately locate the situation where the way the user inputs commands is restricted. After positioning, the voice control video function is automatically turned on, and the video is controlled by voice, which reduces the running time of the program corresponding to the voice control video, reduces the power consumption of the electronic device, and improves the user experience.

[0121] Please refer to Fig. 9 , Fig. 9 1 is a schematic diagram of another method of enabling a voice-controlled video function provided by an embodiment of the present application, which illustrates a process of enabling another electronic device to automatically enable a voice-controlled video function.

[0122] Please refer to Fig. 9 , Fig. 9 The interface 910 shown in (a) is an interface of a video that a user is watching. The interface 910 includes a capsule 911. The capsule 911 refers to a form of an icon corresponding to the automatic start of the voice control video function. Fig. 9 The capsule 911 shown in (a) represents that the automatic voice control video function is in the off state. Fig. 9 During the video playback process of the interface 910 shown in (a), the electronic device determines whether the way in which the user inputs instructions to the electronic device is restricted. For example, the electronic device determines whether the way in which the user inputs instructions to the electronic device is restricted by identifying whether the electronic device is in a non-handheld state. The electronic device can identify whether the electronic device is in a non-handheld state by determining whether a user operation is received within a preset time period, and this application does not limit this.

[0123] When the electronic device determines that the method in which the user inputs instructions to the electronic device is restricted, the electronic device displays Fig. 9 In the interface 920 shown in (b1), the interface 920 displays a pop-up window 912, which is used to prompt the user to enable the automatic voice control video function. Fig. 9 The capsule 911 shown in (b1) in the figure represents that the automatic voice control video function is in an intermediate state, which can be understood as a state between the closed state and the open state. The pop-up window 912 includes a control 913, Fig. 9 The control 913 shown in (b1) is in the closed state.

[0124] Assuming that the user needs to enable the automatic voice control video function, the user can click or slide the control 913. In response to the user's operation of clicking or sliding the control 913, the electronic device can control the control 913 to switch from the closed state to the open state. For example, Fig. 9 The control 913 shown in (b2) is in the turned-on state. Fig. 9 The capsule 911 shown in (b2) in FIG. 1 represents that the automatic voice control video function is turned on. After the control 913 is turned on, the electronic device can cancel the display of the pop-up window 912, and then the user can Figure 5 The illustrated method of inputting instructions is to input instructions to an electronic device.

[0125] In some embodiments, the electronic device Fig. 9 The process of automatically starting the voice control video function can also be: the user can click Fig. 9 In the capsule 911 shown in (a), in response to the user clicking the capsule 911, the electronic device displays Fig. 9 The pop-up window 912 shown in (b1) includes a control 913. Fig. 9 The control 913 shown in (b1) is in a closed state, and the user can click or slide the pop-up window 913. In response to the user clicking or sliding the control 913, the electronic device can control the control 913 to switch from the closed state to the open state. For example, Fig. 9 The control 913 shown in (b2) in the figure is in the turned-on state. After the control 913 is in the turned-on state, the electronic device can cancel the display of the pop-up window 912, and then when the electronic device determines that the way in which the user inputs instructions to the electronic device is restricted, the electronic device can Figure 5 The illustrated method of inputting instructions is to input instructions to an electronic device.

[0126] When the user passes Figure 5 In the process of inputting commands to the electronic device in the manner shown in the input command, the user can turn off the automatic voice control video function in the electronic device. Fig.10 , Fig.10 This is an example diagram of turning off the automatic voice control video function provided by the embodiment of the present application. The user can Fig.10 In (a) of the interface 1010, a sliding operation is performed on the top left side of the screen. In response to the sliding operation performed by the user on the top left side of the screen, the electronic device displays Fig.10 The interface 1020 shown in (b1) in FIG. The interface 1020 includes a control 1021, Fig.10 The control 1021 shown in (b1) is in the open state, and the user can click or slide the control 1021. In response to the user's operation of clicking or sliding the control 1021, the electronic device controls the control 1021 to switch from the open state to the closed state. For example, Fig.10 The control 1021 shown in (b2) is in the closed state. After the control 1021 is in the closed state, the electronic device turns off the automatic voice control video function. Figure 8 Control 811 is also shown in the closed state.

[0127] In some embodiments, when a user passes Figure 5 In the process of inputting a command to the electronic device in the manner shown in the input command, the process of the user turning off the automatic voice control video function in the electronic device can also be that the user Figure 8 In the interface shown in (b2), the control 811 is clicked or slid. In response to the user clicking or sliding the control 811, the electronic device turns off and automatically turns on the voice control video function.

[0128] In an embodiment of the present application, the electronic device can prompt the user when it determines that the way the user inputs commands to the electronic device is restricted, such as prompting the user to enable the automatic voice control video function, thereby improving the user experience.

[0129] The above embodiments introduce a process in which a user can input commands to an electronic device through another method of inputting commands when the method in which a user inputs commands to an electronic device is restricted. The following embodiments introduce a process in which a user can feedback information to a user through another method of feedback information when the method in which an electronic device feeds back information to a user is restricted.

[0130] Please refer to Fig.11 , Fig.11 It is a schematic diagram of another user interacting with an electronic device provided by an embodiment of the present application. It is illustrated that when a user conducts a video conference through an electronic device, the electronic device originally needs to feedback the content of the meeting to the user by playing audio, but due to the noisy environment, the audio volume of the electronic device is too low or muted, the language of the audio played by the electronic device is different from the voice that the user can receive (for example, the electronic device plays English, but the user can only understand Chinese) and other reasons, the original feedback information method of the electronic device is limited, and the user cannot hear the content of the meeting fed back by the electronic device through the above feedback information. The electronic device can feedback information to the user in other ways of feedback information. Other ways of feedback information may be artificial intelligence (AI) subtitles, etc. The embodiment of the present application does not limit the way of other feedback information. The embodiment of the present application takes the way of other feedback information as AI subtitles as an example for explanation.

[0131] for example, Fig.11 The interface 1110 shown in (a) is an interface for a user to conduct a video conference through an electronic device. The interface 1110 includes a floating window 1111. The user can click, slide, or drag a control 1112 in the floating window 1111 to move the control 1112 to a subtitle option. After the control 1112 moves to the subtitle option, the electronic device can display subtitle information corresponding to the audio of the video conference in a text bar 1113 in the floating window 1111.

[0132] Fig.11 (a) in the figure shows that the floating window 1111 also includes a control 1114, and the user can click the control 1114. In response to the user clicking the control 1114, the electronic device displays Fig.11 (b) shows the interface 1120. The interface 1120 includes a floating window 1121, which is a simplified version of the floating window 1111. Compared with the floating window 1111, the floating window 1121 only displays the subtitle information in the text column 1113, and does not display other information in the floating window 1111, such as the controls 1112, 1114 to 1118.

[0133] Fig.11 The floating window 1111 shown in (a) also includes a control 1115. The user can click the control 1115. In response to the user clicking the control 1115, the electronic device can shrink the floating window 1111 into a floating ball. The floating ball can refer to Fig.11 (c) shows an interface 1130 , which includes a suspended ball 1131 .

[0134] Fig.11 The floating window 1111 shown in (a) also includes a control 1116, and the user can drag the control 1116 to move the position of the floating window 1111. For example, the user can drag the control 1116 to move the floating window 1111. Fig.11 The position shown in (a) is moved to Fig.11 The position shown in (d).

[0135] Fig.11 The floating window 1111 shown in (a) of FIG. 1 also includes a control 1117, and the user can adjust the sound source voice, the font size of the subtitle, the opacity of the subtitle background, etc. through the control 1117. For example, the user can click the control 1117, and in response to the user clicking the control 1117, the electronic device displays Fig.12 The interface 1210 shown in (a) of FIG. Fig.12 1210 is an example diagram of an interface 1210 provided in an embodiment of the present application. The interface 1210 includes a floating window 1211, and the floating window 1211 includes a control 1212. The user can adjust the sound source voice through the control 1212. For example, the sound source voice of the video conference is Chinese, and the user can click the control 1212. In response to the user clicking the control 1212, the electronic device displays Fig.12 The interface 1220 shown in (b) of FIG. The interface 1220 includes a floating window 1221, and the floating window includes a control 1222. The user can click the control 1222. In response to the user clicking the control 1222, the electronic device can Fig.11 In the text box 1113 shown in (a) or Fig.11 The floating window 1121 shown in (b) in FIG. 11 shows the Chinese subtitles corresponding to the Chinese sound source speech. In some embodiments, the electronic device can also display the English subtitles corresponding to the English sound source speech, translate the English sound source speech into Chinese subtitles and display the Chinese subtitles corresponding to the English sound source speech, etc., which is not limited in the embodiments of the present application.

[0136] Fig.12 The floating window 1211 shown in (a) of FIG. 1 also includes a control 1213, through which the user can adjust the font size of the subtitles. For example, the font size of the subtitles can be divided into small, standard, large, extra large and extra large, etc. If the user needs to adjust the font size from extra large to standard, the user can click the control 1213. In response to the user clicking the control 1213, the electronic device displays Fig.12 (c) shows an interface 1230. The interface 1230 includes a floating window 1231, and the floating window 1231 includes a slider 1232. The user can drag the slider 1323 to move the slider 1323 to a position corresponding to the standard font size. Fig.12 The position of the slider 1232 is shown in (c). In response to the user dragging the slider 1323, the electronic device can adjust the font size from extra large to standard.

[0137] Continue to refer Fig.11 , Fig.11 The floating window 1111 shown in (a) also includes a control 1118. The user can click on the control 1118, and the electronic device cancels the display of the floating window 1111 in response to the user clicking on the control 1118.

[0138] In some embodiments, some users have hearing impairments. When a user without hearing impairments communicates with a user with hearing impairments, the user without hearing impairments can click, slide, or drag Fig.11 (a) in the figure shows a control 1112 in the floating window 1111, which moves the control 1112 to the dialogue option. After the control 1112 is moved to the dialogue option, the electronic device can display the voice of the user without hearing impairment in the form of text in the floating window 1111, and the user with hearing impairment can understand the content expressed by the other party through the text displayed in the floating window 1111, thereby achieving normal communication between the two parties.

[0139] In the embodiments of the present application, when the way in which the electronic device feeds back information to the user is limited (for example, the way in which the electronic device feeds back information to the user by playing audio is limited), the electronic device can feed back information to the user through other feedback information methods (such as AI subtitles, etc.), which can improve the user experience compared to related solutions in which the electronic device feeds back information to the user in a relatively single way.

[0140] It should be understood that electronic devices Fig.11 Before information is fed back to the user in the form of AI subtitles as shown, the AI ​​subtitle function of the electronic device needs to be turned on. The following is an introduction to the method of turning on the AI ​​subtitle function of the electronic device.

[0141] Please refer to Fig.13 , Fig.13 Schematic diagram of a function of enabling AI subtitles provided by an embodiment of the present application, illustrating a process in which a user manually enables an automatic voice control video function in an electronic device.

[0142] Figure 6 The interface 640 shown in (d) also includes a control 642, and the user can click the control 642. In response to the user clicking the control 642, the electronic device displays Fig.13 The interface 1310 shown in (a1) in FIG. The interface 1310 includes a control 1311, Fig.13 The control 1311 shown in (a1) is in a closed state. The user can click or slide the control 1311, and the electronic device can switch the control 1311 from the closed state to the open state. For example, Fig.13 The control 1311 shown in (a2) is in the turned-on state. After the control 1311 is in the turned-on state, the electronic device can Fig.11 The AI ​​subtitles shown provide feedback information to the user.

[0143] Please refer to Fig.14 , Fig.14 1 is a schematic diagram of another method of enabling AI subtitles provided by an embodiment of the present application, which illustrates another process of a user manually enabling AI subtitles in an electronic device.

[0144] Users can Fig.14 In the interface 1410 shown in (a), a sliding operation is performed on the top right side of the screen. In response to the sliding operation of the user, the electronic device displays Fig.14 The interface 1420 shown in (b1) includes a control 1421. Fig.14 The control 1421 shown in (b1) is in a closed state, and the user can click the control 1421. In response to the user clicking the control 1421, the electronic device controls the control 1421 to switch from the closed state to the open state. For example, Fig.14 The control 1421 shown in (b2) is in the turned-on state. After the control 1421 is in the turned-on state, the electronic device can Fig.11 The AI ​​subtitles shown provide feedback information to the user.

[0145] By the above Fig.13 and Fig.14 It can be seen that the electronic device in the embodiment of the present application provides a variety of controls for turning on the AI ​​subtitles function. The user can turn on the AI ​​subtitles function in a variety of ways. When the user has not mastered or forgotten one way to turn on the AI ​​subtitles function, the user can turn on the AI ​​subtitles function in another way, thereby improving the user experience.

[0146] Please refer to Fig.15 , Fig.15 1 is a schematic diagram of another method of enabling the AI ​​subtitles function provided by an embodiment of the present application, which illustrates a process of enabling the automatic enabling of the AI ​​subtitles function in an electronic device.

[0147] Users can click Figure 6 In the control 642 shown in (d), in response to the user clicking the control 642, the electronic device displays Fig.15 The interface 1510 shown in (a1) includes a control 1511. Fig.15 The control 1511 shown in (a1) is in the closed state. The user can click on the control 1511. In response to the user's operation of clicking on the control 1511, the electronic device controls the control 1511 to switch from the closed state to the open state. For example, Fig.15 The control 1511 shown in (a2) is in the turned-on state.

[0148] After the control 1511 is turned on, when the user conducts a video conference through the electronic device, when the electronic device determines that the way to feedback information to the user is restricted (for example, when the electronic device determines that the way to feedback information to the user through audio is restricted), the electronic device can automatically turn on the AI ​​subtitle function. After the AI ​​subtitle function is automatically turned on, the electronic device can Fig.11 The information is fed back to the user in the manner shown in (a) or (b).

[0149] The implementation process of the electronic device determining whether the manner of feeding back information to the user is restricted may be:

[0150] Step 1: The electronic device determines whether a video application is running.

[0151] It should be understood that video applications refer to apps that provide functions such as video playback, video conferencing, and video calls.

[0152] In the embodiments of the present application, the implementation process of the electronic device determining whether a video application is running can refer to the above embodiments and will not be repeated here.

[0153] In some embodiments, the electronic device can also determine whether a video application is running by calling a method to determine whether the current device is playing audio or video. For example, the method to determine whether the current device is playing audio or video can be isMusicOrVideoPlay(), etc., and the embodiments of the present application are not limited to this.

[0154] Step 2: When it is determined that the video application is running, determine whether the video content presented by the video application has human voice audio.

[0155] In implementation, the electronic device can determine whether the video content presented by the video application has human voice audio through a human voice recognition algorithm. The embodiment of the present application does not limit the method of determining whether the video content presented by the video application has human voice audio.

[0156] Step 3: When the electronic device determines that the video presented by the video application contains human voice audio, it determines whether the video content presented by the video application has subtitle information.

[0157] In implementation, the electronic device may determine whether the video content presented by the video application has subtitle information by detecting whether the video file has subtitle information, which is not limited in this embodiment of the present application.

[0158] Step 4: When the electronic device determines that the video content presented by the video application does not have subtitle information, the electronic device determines whether the volume value of the electronic device is less than a preset volume value.

[0159] It should be understood that the preset volume value may be 10 decibels, 15 decibels, etc.

[0160] In the implementation, the electronic device can obtain the volume value through the AudioManager class in Android.

[0161] Step 5: When the electronic device determines that the volume value of the electronic device is less than the preset volume value, the electronic device determines whether the manner in which the electronic device feeds back information to the user is restricted.

[0162] It should be understood that if the volume value of an electronic device is less than the preset volume value, it means that the volume of the electronic device is too low or muted, and the user can no longer hear the video content presented by the electronic device at this volume. It should also be understood that when a video application presents video content with human voice audio but no subtitle information on the display screen of an electronic device, assuming that the volume of the electronic device is too low or muted, the user cannot hear the video content that the electronic device originally fed back to the user by playing human voice audio, and the way in which the electronic device feeds back information to the user is limited. For example, when a user is having a meeting through a video conference, the video conference has human voice audio but does not have subtitle information corresponding to the human voice audio, and the volume of the electronic device is too low or muted, then the way in which the electronic device feeds back information to the user is limited.

[0163] In some embodiments, the method for determining whether the manner in which the electronic device feeds back information to the user is restricted may also be:

[0164] After the above step 3, the electronic device executes step 6.

[0165] Step 6: When the electronic device determines that the video content presented by the video application does not have subtitle information, the electronic device determines whether the noise level of the surrounding environment of the electronic device is greater than a preset noise level.

[0166] It should be understood that the preset noise level may be 50 decibels, 60 decibels, etc., and this is not limited in the embodiments of the present application.

[0167] It should also be understood that if the noise level of the environment around the electronic device is greater than the preset noise level, it means that the user can no longer hear the video content presented by the electronic device in this environment.

[0168] In implementation, the electronic device may determine the noise level of the environment surrounding the electronic device by using a tool for measuring the noise level, which is not limited in the embodiments of the present application.

[0169] Step 7: When the electronic device determines that the noise level of the environment around the electronic device is greater than a preset noise level, the electronic device determines that the manner in which the electronic device feeds back information to the user is restricted.

[0170] It should be understood that when a video application presents video content with human voice audio but without subtitle information on the display screen of an electronic device, assuming that the noise level in the environment surrounding the electronic device is too high, the user cannot clearly hear the video content that the electronic device originally fed back to the user by playing human voice audio, and the way in which the electronic device feeds back information to the user is limited.

[0171] In some embodiments, the method for determining whether the manner in which the electronic device feeds back information to the user is restricted may also be:

[0172] After the above step 3, the electronic device executes step 8.

[0173] Step 8: When the electronic device determines that the video content presented by the video application does not have subtitle information, the electronic device determines whether the audio language corresponding to the video content is the same as the system language of the electronic device.

[0174] It should be understood that if the audio language corresponding to the video content is English and the system language of the electronic device is Chinese, then the audio language corresponding to the video content is different from the system language of the electronic device. If the audio language corresponding to the video content is Chinese and the system language of the electronic device is Chinese, then the audio language corresponding to the video content is the same as the system language of the electronic device.

[0175] It should also be understood that if the audio language corresponding to the video content is the same as the system language of the electronic device, it means that the user can understand the audio language corresponding to the video content. If the audio language corresponding to the video content is different from the system language of the electronic device, it means that the user cannot understand the audio language corresponding to the video content.

[0176] In implementation, the electronic device may obtain the system language through the Locale class, and the electronic device may identify the audio language corresponding to the video content through a language recognition algorithm, which is not limited in this application.

[0177] Step 9: When the electronic device determines that the audio language corresponding to the video content is different from the system language of the electronic device, it determines that the way in which the electronic device feeds back information to the user is restricted.

[0178] It should be understood that when a video application presents video content with human voice audio but without subtitle information on the display screen of an electronic device, assuming that the audio language corresponding to the video content is different from the system language of the electronic device, the user cannot understand the audio language corresponding to the video content that the electronic device originally fed back to the user by playing human voice audio, and the way in which the electronic device feeds back information to the user is limited.

[0179] In the embodiment of the present application, the electronic device can enable the automatic opening of the AI ​​subtitle function when the electronic device accurately locates that the way to feedback information to the user through audio is limited, and through the above Fig.13 and Fig.14 After the manual AI subtitle function is enabled, the electronic device can feedback information to the user through AI subtitles when the way the electronic device feeds back information to the user through audio is restricted or not restricted. That is to say, the program corresponding to the AI ​​subtitles is running, which increases the power consumption of the electronic device. The embodiment of the present application can accurately locate the situation where the way the electronic device feeds back information is restricted, and automatically turn on the AI ​​subtitle function after positioning, thereby reducing the running time of the program corresponding to the AI ​​subtitles, reducing the power consumption of the electronic device, and improving the user experience.

[0180] Please refer to Fig.16 , Fig.16 This is a schematic diagram of an embodiment of the present application that prompts a user to enable a function of automatically turning on AI subtitles. Fig.16 The interface 1610 shown in (a) is an interface for a user to conduct a video conference through an electronic device. The interface 1610 includes a capsule 1611. The capsule 1611 refers to a form of an icon corresponding to the function of the AI ​​subtitle. Fig.16 The capsule 1611 shown in (a) in FIG. 1 represents that the AI ​​subtitle function is in the off state.

[0181] Users through Fig.16 During the video conference in the interface 1610 shown in (a), the electronic device determines whether the way of feeding back information to the user is restricted. For example, the electronic device determines whether the way of feeding back information to the user is restricted by identifying whether the video with voice has subtitles, identifying the volume of the media of the electronic device, etc. The implementation process of the electronic device determining whether the way of feeding back information to the user is restricted can be referred to the following embodiment, which will not be described here.

[0182] When the electronic device determines that the way in which the electronic device provides feedback to the user is limited, the electronic device displays Fig.16 The interface 1620 shown in (b1) in the figure displays a pop-up window 1612 . Fig.16 The capsule 1611 shown in (b1) in FIG. 1 represents that the function of the AI ​​subtitle is in an intermediate state, which can be understood as a state between the closed state and the open state. The pop-up window 1612 includes a control 1613, Fig.16 The control 1613 shown in (b1) is in the closed state.

[0183] Assuming that the user needs to enable the function of automatically turning on AI subtitles, the user can click or slide the control 1613. In response to the user's operation of clicking or sliding the control 1613, the electronic device can control the control 1613 to switch from the off state to the on state. For example, Fig.16 The control 1613 shown in (b2) is in the turned-on state. Fig.16 The capsule 1611 shown in (b2) in FIG. 1 represents that the function of automatically turning on AI subtitles is in the turned-on state. Fig.16 (b1) in the figure shows that the pop-up window 1612 also includes text 1614. The user can click on the text 1614 to view more ways for the electronic device to determine whether the way the electronic device feeds back information to the user is restricted. For example, the electronic device can determine whether the way the electronic device feeds back information to the user is restricted by identifying whether the audio language of the video with voice is different from the system language of the electronic device, identifying the noise intensity when the electronic device plays the video with voice, etc. After the control 1613 is turned on, the electronic device can cancel the display of the pop-up window 1612, and then the electronic device can Fig.11 The AI ​​subtitles shown in (a) or (b) provide feedback information to the user.

[0184] In some embodiments, the electronic device Fig.16 The process of enabling the automatic opening of the AI ​​subtitle function can also be: the user can click Fig.16 (a) shows a capsule 1611. In response to the user clicking the capsule 1611, the electronic device displays Fig.16 The pop-up window 1612 shown in (b1) of FIG. 1 includes a control 1613. The user can click or slide the control 1613. In response to the user clicking or sliding the control 1613, the electronic device can control the control 1613 to switch from a closed state to an open state. After the control 1613 is in the open state, the electronic device can cancel the display of the pop-up window 1612. After that, when the electronic device determines that the way to feedback information to the user is restricted, the electronic device can Fig.11 The AI ​​subtitles shown in (a) or (b) provide feedback information to the user.

[0185] In electronic devices through Fig.11 In the process of providing feedback to the user in the manner shown in the feedback information, the user can turn off the function of automatically turning on AI subtitles in the electronic device. For example, please refer to Fig.17 , Fig.17 This is an example diagram of a function of turning off AI subtitles provided by an embodiment of the present application. Users can Fig.17 In the interface 1710 shown in (a), a sliding operation is performed on the top left side of the screen. In response to the sliding operation performed by the user on the top left side of the screen, the electronic device displays Fig.17 The interface 1720 shown in (b1) in FIG. The interface 1720 includes a control 1721, Fig.17 The control 1721 shown in (b1) is in the on state, and the user can click or slide the control 1721. In response to the user clicking or sliding the control 1721, the electronic device controls the control 1721 to switch from the on state to the off state. After the control 1721 is in the off state, the electronic device turns off the function of automatically turning on the AI ​​subtitles. Fig.15 Control 1511 is also shown in the closed state.

[0186] In some embodiments, the electronic device Fig.11 In the process of providing feedback to the user in the manner of feedback information shown, the user can turn off the function of automatically turning on AI subtitles in the electronic device. The process can also be that the user Fig.15 In the interface shown, click or slide control 1511. In response to the user clicking or sliding control 1511, the electronic device turns off the automatically turned-on AI subtitle function.

[0187] In an embodiment of the present application, the electronic device can prompt the user when it determines that the way the user inputs commands to the electronic device is restricted, such as prompting the user to enable the automatic voice control video function, thereby improving the user experience.

[0188] The above embodiments introduce a process in which a user can input a command to an electronic device through another method of inputting a command when the method in which a user inputs a command to an electronic device is restricted, and introduce a process in which an electronic device can feedback information to a user through another method of feedback information when the method in which an electronic device feeds back information to a user is restricted. The following embodiments introduce an implementation process in which an electronic device determines whether the method in which a user inputs a command to an electronic device is restricted.

[0189] The following introduces an implementation process of a control method for an electronic device provided in an embodiment of the present application.

[0190] The method includes: displaying a first interface, the first interface including a first control; in response to an operation on the first control, opening a voice control video function of the electronic device; displaying a second interface, the second interface displaying a video; and controlling the video displayed on the second interface according to a voice command;

[0191] Or; display a second interface; detect user operations on the electronic device; if no user operations are detected within a preset time, control the video displayed on the second interface based on the voice command and automatically start the voice control video function.

[0192] It should be understood that the first interface can refer to Figure 6 In the interface 650 shown in (e1), the first control can refer to the control 651 in the interface 650. The second interface can refer to Figure 5 The interface shown.

[0193] In the implementation, users can Figure 6 The control 651 in the interface 650 shown in (e1) is operated, and in response to the user's operation on the control 651, the voice control video function of the electronic device is turned on.

[0194] After it is turned on, users can Figure 5 When watching a video in the interface shown, you can control the video by voice. For example, Figure 5 The user's voice command shown in (a) is to fast forward 10 seconds. The electronic device can control the video to fast forward 10 seconds according to the voice command. Other ways to control the video by voice can refer to Figure 5 (b) and (c) in the present application are not limited to this.

[0195] It should also be understood that the implementation process of the electronic device detecting the user operation on the electronic device and failing to detect the user operation within a preset time period can refer to the above embodiments and will not be repeated here.

[0196] In a related scheme, when a user uses an electronic device with a relatively simple interaction method to watch a video played by the electronic device, the user can input instructions to the electronic device through operations such as clicking and touching to control the video played by the electronic device. For example, the user can adjust the video played by the electronic device by clicking the volume control of the electronic device, and the user can touch the display screen of the electronic device to pause the video being played by the electronic device. However, the above-mentioned method of inputting instructions to the electronic device may be restricted. For example, while the electronic device is playing a video, the user is handling other matters (such as cooking, sweeping the floor, etc.), it is inconvenient for the user to input instructions to the electronic device through operations such as clicking and touching, which affects the user experience.

[0197] In an embodiment of the present application, the electronic device can enable the voice control video function of the electronic device through the first control included in the first interface. After the function is enabled, when the electronic device displays the second interface (the electronic device plays a video on the second interface), when the user is restricted from inputting commands to the electronic device through operations such as clicking and touching, the user can input voice commands to the electronic device in the form of voice, and the electronic device can control the played video according to the user's voice commands, thereby improving the user experience.

[0198] In an embodiment of the present application, when the electronic device displays the second interface (the electronic device plays a video on the second interface), it can detect the user's operation on the electronic device (the user inputs instructions to the electronic device through user operations (such as clicking, touching, etc.)). If no user operation is detected within a preset time period (this situation proves that the way in which the user inputs instructions to the electronic device through user operation is limited), and it is inconvenient for the user to input instructions to the electronic device through user operation, the electronic device can control the played video based on the user's voice command and automatically start the voice control video function (the automatic voice control video function can be turned on before detecting the user's operation on the electronic device, and can also be turned on when no user operation is detected within a preset time period), thereby improving the user experience.

[0199] In some embodiments, before displaying the first interface, the method also includes: displaying a third interface, the third interface including a first icon; in response to an operation on the first icon, displaying a fourth interface, the fourth interface including a second control; in response to an operation on the second control, displaying a fifth interface, the fifth interface including a third control; in response to an operation on the third control, displaying a sixth interface, the sixth interface including a fourth control; and displaying the first interface, including: displaying the first interface in response to an operation on the fourth control.

[0200] It should be understood that the third interface can refer to Figure 6 In the interface 610 shown in (a), the first icon can refer to Figure 6 The control 611 shown in (a) of FIG. 4 can refer to Figure 6 The interface 620 shown in (b) of FIG. 1 , the second control can refer to the control 621 shown in the interface 620, and the fifth interface can refer to Figure 6 In the interface 630 shown in (c), the third control can refer to the control 631 in the interface 630, and the sixth interface can refer to Figure 6 In the interface 640 shown in (d), the fourth control can refer to the control 641 in the interface 640.

[0201] Regarding the process of the electronic device displaying the first interface in the embodiment of the present application, reference can be made to the above embodiment and will not be repeated here.

[0202] In an embodiment of the present application, the electronic device can display a first interface in response to a user's operation on a first icon, a second control, a third control, or a fourth control, and can enable a voice control video function in response to a user's operation on a first control in the first interface. Through the above operations, the user can quickly and conveniently enable the voice control video function of the electronic device, thereby improving the user experience.

[0203] In some embodiments, before displaying the first interface, the method further includes:

[0204] Displaying a fourth interface, the fourth interface including a fifth control; in response to an operation on the fifth control, displaying a seventh interface, the seventh interface including a sixth control; and displaying the first interface, including: displaying the first interface in response to an operation on the sixth control.

[0205] It should be understood that the fourth interface can refer to Figure 6 The interface 620 shown in (b) of FIG. 5 , the fifth control can refer to the control 622 in the interface 620, and the seventh interface can refer to Figure 7 In the interface 710 shown in (a1), the sixth control can refer to the control 711 in the interface 710.

[0206] Regarding the process of the electronic device displaying the first interface in the embodiment of the present application, reference can be made to the above embodiment and will not be repeated here.

[0207] An embodiment of the present application provides another way for a user to turn on a control video function. Specifically, the electronic device can display a first interface in response to the user's operation on a fifth control and an operation on a sixth control, and can turn on a voice control video function in response to the user's operation on a first control in the first interface. The user can turn on the voice control video function of the electronic device through the above operations. The electronic device provides the user with a variety of ways to turn on the voice control video function of the electronic device, thereby improving the user experience.

[0208] In some embodiments, the second interface includes a first capsule; and before detecting a user operation on the electronic device, the method further includes:

[0209] In response to the operation on the first capsule, a first pop-up window is displayed, and the first pop-up window includes a seventh control; in response to the operation on the seventh control, the automatic voice control video function is turned on.

[0210] It should be understood that the first capsule can refer to Fig. 9 The capsule 911 shown in (a) of FIG. 1 can be referred to as the first pop-up window. Fig. 9 In the pop-up window 912 shown in (b1), the seventh control can refer to the control 913 in the pop-up window 912.

[0211] Regarding the implementation process of automatically turning on the voice control video function in the embodiments of the present application, reference can be made to the above embodiments and will not be repeated here.

[0212] In an embodiment of the present application, before detecting a user operation on the electronic device, the electronic device can respond to the user's operation on the first capsule and the operation on the seventh control in the first pop-up window to turn on the automatic voice control video function. Through the above operations, the user can quickly and conveniently enable the automatic voice control video function of the electronic device to improve the user experience.

[0213] In some embodiments, when no user operation is detected within a preset time period, the video displayed on the second interface is controlled based on the automatic start of the voice control video function according to the voice instruction, including: when no user operation is detected within the preset time period, a first pop-up window is displayed, and the first pop-up window includes a seventh control; in response to the operation of the seventh control, the automatic start of the control video function is turned on; according to the voice instruction, the video displayed on the second interface is controlled.

[0214] It should be understood that the seventh control can refer to Fig. 9 (b1) shows the control 913 in the pop-up window 912.

[0215] The implementation process of the electronic device not detecting a user operation within a preset time period can refer to the above embodiment and will not be repeated here.

[0216] The implementation process of turning on the automatic control video function and controlling the video displayed on the second interface according to voice commands in the embodiments of the present application can refer to the above embodiments and will not be repeated here.

[0217] In an embodiment of the present application, the electronic device can prompt the user through a first pop-up window to enable the automatic start of voice control video function when no user operation is detected within a preset time period. Specifically, the electronic device can prompt the user to enable the automatic start of voice control video function through the seventh control in the first pop-up window. The user can enable the automatic start of voice control video function according to the above prompt, thereby improving the user experience.

[0218] The following introduces an implementation process of a control method for an electronic device provided in an embodiment of the present application.

[0219] The method includes: displaying an eighth interface, the eighth interface including an eighth control; in response to an operation on the eighth control, enabling a subtitle function of the electronic device; or: displaying a third interface; in response to an operation on the third interface, displaying a ninth interface, the ninth interface including a ninth control; in response to an operation on the ninth control, enabling a subtitle function of the electronic device; displaying a second interface, the second interface displaying a video; and displaying subtitles corresponding to the voice of the video in the second interface;

[0220] Or; display a second interface, the second interface displays a video; detect the video file corresponding to the video: when it is detected that the video file does not include subtitle information, based on the automatic activation of the subtitle function, the subtitles corresponding to the voice of the video are displayed in the second interface.

[0221] It should be understood that the eighth interface can refer to Fig.13 The interface 1310 shown in (a1) in FIG. 8 , the eighth control can refer to the control 1311 in the interface 1310, and the third interface can refer to Fig.14 In the interface 1410 shown in (a), the operation on the third interface can be a sliding operation performed by the user on the top right side of the screen of the interface 1410. The ninth interface can refer to Fig.14 The ninth control can refer to the control 1421 in the interface 1420. The second interface can refer to Fig.11 The interface shown in .

[0222] The electronic device can refer to the above embodiment for the implementation process of displaying the subtitles corresponding to the voice of the video in the second interface, and will not be repeated here. The electronic device can refer to the above embodiment for the implementation process of detecting the video file corresponding to the video and detecting that the video file does not include subtitle information, and will not be repeated here.

[0223] In a related solution, when a user uses a video function of an electronic device with a relatively simple interaction method (for example, the user uses the electronic device to conduct a video conference), the electronic device can provide feedback to the user on the content of the video conference by playing the voice of the video. However, the manner in which the above-mentioned electronic device provides feedback to the user may be limited. For example, if the noise intensity of the surrounding environment of the electronic device is too high, or the volume of the voice of the video played by the electronic device is too low or muted, the user cannot hear the voice of the video clearly, which affects the user experience.

[0224] In an embodiment of the present application, the electronic device can enable the subtitle function of the electronic device through the eighth control in the eighth interface, or through the third interface and the ninth control in the ninth interface. After the function is turned on, when the user uses the video function of the electronic device (the second interface displays a video, and the user conducts a video conference through the video displayed on the second interface), when the electronic device is limited in the way of providing feedback to the user through the voice of the video being played, the electronic device can provide feedback to the user by displaying subtitles corresponding to the voice of the video in the second interface, thereby improving the user experience.

[0225] In an embodiment of the present application, when displaying the second interface, the electronic device can detect the video file corresponding to the video. When it is detected that the video file does not include subtitle information (this situation proves that the way in which the electronic device feeds back information to the user through the voice of the video is limited), the electronic device can display the subtitles corresponding to the voice of the video in the second interface based on the automatic subtitle activation function (the automatic subtitle activation function can be activated before detecting the video file corresponding to the video, and can also be activated when it is detected that the video file does not include subtitle information), thereby improving the user experience.

[0226] In some embodiments, before displaying the eighth interface, the method also includes: displaying a third interface, the third interface including a first icon; in response to an operation on the first icon, displaying a fourth interface, the fourth interface including a second control; in response to an operation on the second control, displaying a fifth interface, the fifth interface including a third control; in response to an operation on the third control, displaying a sixth interface, the sixth interface including a tenth control; and, displaying the eighth interface, including: displaying the eighth interface in response to an operation on the tenth control.

[0227] It should be understood that the explanations of the third interface, the first icon, the fourth interface, the second control, the fifth interface, the third control, and the sixth interface can refer to the above embodiments, and the tenth control can refer to Figure 6 Control 642 is shown in (d).

[0228] The process of the electronic device displaying the eighth interface in the embodiment of the present application can refer to the above embodiment and will not be repeated here.

[0229] In the embodiment of the present application, the electronic device can display the eighth interface in response to the operation of the first icon, the operation of the second control, the operation of the third control, and the operation of the tenth control, and can enable the subtitle function in response to the user's operation of the eighth control in the eighth interface. The user can use the above operations to quickly and conveniently enable the subtitle function of the electronic device, thereby improving the user experience.

[0230] In some embodiments, the second interface includes a second capsule; and, before detecting the video file corresponding to the video, the method also includes: in response to the operation of the second capsule, displaying a second pop-up window, the second pop-up window including an eleventh control; in response to the operation of the eleventh control, turning on the automatic subtitle opening function.

[0231] It should be understood that the second capsule can refer to Fig.16 The capsule 1611 shown in (a) of FIG. 1 and the second pop-up window can refer to Fig.16 In the pop-up window 1612 shown in (b), the eleventh control can refer to the control 1613 in the pop-up window 1612.

[0232] The process of enabling the automatic subtitle activation function of the electronic device in the embodiment of the present application can refer to the above embodiment and will not be repeated here.

[0233] In an embodiment of the present application, before the electronic device detects the video file corresponding to the video, the electronic device can respond to the operation of the second capsule and the operation of the eleventh control in the second pop-up window to turn on the automatic subtitle opening function. Through the above operations, the user can quickly and conveniently enable the automatic subtitle opening function of the electronic device to improve the user experience.

[0234] In some embodiments, when it is detected that the video file does not include subtitle information, based on the automatic subtitle opening function, subtitles corresponding to the voice of the video are displayed in the second interface, including: when it is detected that the video file does not include subtitle information, a second pop-up window is displayed, and the second pop-up window includes an eleventh control; in response to the operation of the eleventh control, the automatic subtitle opening function is turned on; and subtitles corresponding to the voice of the video are displayed in the second interface.

[0235] It should be understood that the second capsule can refer to Fig.16 The capsule 1611 shown in (a) of FIG. 1 and the second pop-up window can refer to Fig.16 In the pop-up window 1612 shown in (b), the eleventh control can refer to the control 1613 in the pop-up window 1612.

[0236] The implementation process of the electronic device detecting that the video file does not include subtitle information in the embodiment of the present application can refer to the above embodiment, which will not be repeated here. The implementation process of the electronic device turning on the automatic subtitle function can refer to the above embodiment, which will not be repeated here.

[0237] In an embodiment of the present application, the electronic device can prompt the user through a second pop-up window to enable the automatic subtitle opening function when it detects that the video file does not include subtitle information. Specifically, the electronic device can prompt the user to enable the automatic subtitle opening function through an eleventh control in the second pop-up window. The user can enable the automatic subtitle opening function according to the above prompt, thereby improving the user experience.

[0238] In some embodiments, detecting a video file corresponding to a video includes: detecting subtitle information in the video file; obtaining a volume value when the electronic device plays the video when detecting that the video file does not include subtitle information; displaying subtitles corresponding to the voice of the video in a second interface based on an automatic subtitle function when detecting that the video file does not include subtitle information, including: displaying subtitles corresponding to the voice of the video in a second interface based on an automatic subtitle function when determining that the volume value when the electronic device plays the video is less than a preset volume value.

[0239] The implementation process of obtaining the volume value when the electronic device plays a video and determining that the volume value when the electronic device plays a video is less than the preset volume value in the embodiment of the present application can refer to the above embodiment and will not be repeated here.

[0240] In an embodiment of the present application, the electronic device can quickly and accurately locate the scenario where the way in which the electronic device feeds back information to the user through the voice of the video is restricted by detecting that the video file does not include subtitle information and determining that the volume value of the electronic device when playing the video is less than a preset volume value. After locating the above scenario, the electronic device can automatically turn on the subtitle function and feed back information to the user by displaying subtitles corresponding to the voice of the video in the second interface, instead of feeding back information to the user by displaying subtitles corresponding to the voice of the video during the entire process of the user using the video function of the electronic device, thereby reducing the time the electronic device displays subtitles and reducing the power consumption of the electronic device.

[0241] In some embodiments, detecting a video file corresponding to a video includes: detecting subtitle information in the video file; obtaining a noise value of a current environment in which the electronic device is located when detecting that the video file does not include subtitle information; displaying subtitles corresponding to the voice of the video in a second interface based on an automatic subtitle function when detecting that the video file does not include subtitle information, including: displaying subtitles corresponding to the voice of the video in a second interface based on an automatic subtitle function when determining that the noise value of the current environment in which the electronic device is located is less than a preset noise value.

[0242] The implementation process of obtaining the noise value of the current environment in which the electronic device is located and determining that the noise value of the current environment in which the electronic device is located is less than the preset noise value in the embodiment of the present application can refer to the above embodiment and will not be repeated here.

[0243] In an embodiment of the present application, the electronic device can quickly and accurately locate the scenario where the electronic device is restricted from providing feedback to the user through the voice of the video being played by detecting that the video file does not include subtitle information and determining that the noise value of the current environment in which the electronic device is located is less than a preset noise value. After locating the above scenario, the electronic device can provide feedback to the user by displaying subtitles corresponding to the voice of the video in a second interface based on automatically turning on the subtitle function, without providing feedback to the user by displaying subtitles corresponding to the voice of the video during the entire process of the user using the video function of the electronic device, thereby reducing the time the electronic device displays subtitles and reducing the power consumption of the electronic device.

[0244] In some embodiments, detecting a video file corresponding to a video includes: detecting subtitle information in the video file; when it is detected that the video file does not include subtitle information, obtaining an audio language corresponding to the video; and when it is detected that the video file does not include subtitle information, based on automatically starting a subtitle function, displaying subtitles corresponding to the voice of the video in a second interface, including: when it is determined that the audio language corresponding to the video is different from the system language of the electronic device, based on automatically starting a subtitle function, displaying subtitles corresponding to the voice of the video in a second interface.

[0245] The implementation process of obtaining the audio language corresponding to the video and determining that the audio language corresponding to the video is different from the system language of the electronic device in the embodiment of the present application can refer to the above embodiment and will not be repeated here.

[0246] In an embodiment of the present application, the electronic device can quickly and accurately locate the scenario where the way in which the electronic device feeds back information to the user through the voice of the video is restricted by detecting that the video file does not include subtitle information and determining that the audio language corresponding to the video is different from the system language of the electronic device. After locating the above scenario, the electronic device can automatically turn on the subtitle function and feed back information to the user by displaying the subtitles corresponding to the voice of the video in the second interface, instead of feeding back information to the user by displaying the subtitles corresponding to the voice of the video during the entire process of the user using the video function of the electronic device, thereby reducing the time the electronic device displays subtitles and reducing the power consumption of the electronic device.

[0247] It should be understood that the user operation in the above-mentioned embodiments may be a user's click operation, sliding operation, gesture sensing operation, or voice control, etc. The specific form of the user operation in the embodiments of the present application is not limited.

[0248] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0249] The embodiment of the present application provides a computer program product, when the computer program product is run on an electronic device, the electronic device executes the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above method-related embodiments, and will not be repeated here.

[0250] The embodiment of the present application provides a readable storage medium, the readable storage medium contains instructions, when the instructions are executed in an electronic device, the electronic device executes the technical solution of the above embodiment. The implementation principle and technical effect are similar, and will not be repeated here.

[0251] The embodiment of the present application provides a chip, the chip is used to execute instructions, when the chip is running, the technical solution in the above embodiment is executed. The implementation principle and technical effect are similar, and will not be repeated here.

[0252] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0253] It should be understood that the "embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the various embodiments in the entire specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0254] A person skilled in the art may understand that the various numerical numbers such as first and second involved in the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application, and also indicate the order of precedence.

[0255] In this application, elements expressed in the singular are intended to mean "one or more" rather than "one and only one", unless otherwise specified. In this application, "at least one" is intended to mean "one or more", and "more than one" is intended to mean "two or more", unless otherwise specified.

[0256] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. A can be singular or plural, and B can be singular or plural.

[0257] In this document, the term "at least one of..." or "at least one of..." means all or any combination of the listed items. For example, "at least one of A, B and C" may mean: A exists alone, B exists alone, C exists alone, A and B exist at the same time, B and C exist at the same time, and A, B and C exist at the same time, where A may be singular or plural, B may be singular or plural, and C may be singular or plural.

[0258] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0259] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0260] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0261] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0262] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0263] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0264] The same or similar parts between the various embodiments in this application can refer to each other. In the various embodiments in this application, and the various implementation methods / implementation methods / implementation methods in each embodiment, if there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments and the various implementation methods / implementation methods / implementation methods in each embodiment are consistent and can be referenced to each other. The technical features in different embodiments and the various implementation methods / implementation methods / implementation methods in each embodiment can be combined to form new embodiments, implementation methods, implementation methods, or implementation methods according to their inherent logical relationships. The above-described implementation methods of this application do not constitute a limitation on the scope of protection of this application.

[0265] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims. In short, the above is only a preferred embodiment of the technical solution of the present application, and is not used to limit the protection scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.< / videoview> < / imgview> < / textview>

Claims

1. A control method for an electronic device, characterized in that: include: Display a first interface, wherein the first interface includes a first control; In response to the operation of the first control, a voice control video function of the electronic device is enabled; Displaying a second interface, wherein the second interface displays a video; Controlling the video displayed on the second interface according to the voice command; or; displaying the second interface; detecting a user operation on the electronic device; When the user operation is not detected within a preset time period, the video displayed on the second interface is controlled based on automatically starting the voice control video function according to the voice instruction.

2. The method according to claim 1, characterized in that Before displaying the first interface, the method further includes: Displaying a third interface, wherein the third interface includes the first icon; In response to an operation on the first icon, displaying a fourth interface, wherein the fourth interface includes a second control; In response to an operation on the second control, displaying a fifth interface, wherein the fifth interface includes a third control; In response to the operation on the third control, displaying a sixth interface, wherein the sixth interface includes the fourth control; Displaying the first interface includes: In response to an operation on the fourth control, the first interface is displayed.

3. The method according to claim 1, characterized in that Before displaying the first interface, the method further includes: Displaying a fourth interface, wherein the fourth interface includes a fifth control; In response to the operation on the fifth control, displaying a seventh interface, wherein the seventh interface includes the sixth control; Displaying the first interface includes: In response to an operation on the sixth control, the first interface is displayed.

4. The method according to claim 1, characterized in that The second interface includes a first capsule; and before detecting a user operation on the electronic device, the method further includes: In response to the operation on the first capsule, displaying a first pop-up window, wherein the first pop-up window includes a seventh control; In response to the operation of the seventh control, the automatic voice control video function is turned on.

5. The method according to claim 1, characterized in that When the user operation is not detected within the preset time, controlling the video displayed on the second interface based on the automatic start of the voice control video function according to the voice instruction includes: When the user operation is not detected within a preset time period, displaying a first pop-up window, wherein the first pop-up window includes a seventh control; In response to the operation of the seventh control, the automatic voice control video function is turned on; The video displayed on the second interface is controlled according to the voice command.

6. A method for controlling an electronic device, characterized in that: include: Displaying an eighth interface, wherein the eighth interface includes an eighth control; In response to the operation of the eighth control, a subtitle function of the electronic device is enabled; or; Display the third interface; In response to an operation on the third interface, displaying the ninth interface, wherein the ninth interface includes a ninth control; In response to the operation of the ninth control, the subtitle function of the electronic device is turned on; Displaying a second interface, wherein the second interface displays a video; Displaying subtitles corresponding to the voice of the video in the second interface; or; Displaying the second interface, where the second interface displays a video; Detecting a video file corresponding to the video; When it is detected that the video file does not include subtitle information, based on the automatic activation of the subtitle function, the subtitles corresponding to the voice of the video are displayed in the second interface.

7. The method according to claim 6, characterized in that Before displaying the eighth interface, the method further includes: Displaying a third interface, wherein the third interface includes the first icon; In response to an operation on the first icon, displaying a fourth interface, wherein the fourth interface includes a second control; In response to an operation on the second control, displaying a fifth interface, wherein the fifth interface includes a third control; In response to the operation on the third control, displaying a sixth interface, wherein the sixth interface includes a tenth control; The eighth interface is displayed, including: In response to the operation on the tenth control, the eighth interface is displayed.

8. The method according to claim 6, characterized in that The second interface includes a second capsule; and before detecting the video file corresponding to the video, the method further includes: In response to the operation on the second capsule, displaying a second pop-up window, wherein the second pop-up window includes an eleventh control; In response to the operation of the eleventh control, the automatic subtitle opening function is turned on.

9. The method according to claim 6, characterized in that The method of, in the case where it is detected that the video file does not include subtitle information, displaying subtitles corresponding to the voice of the video in the second interface based on the automatic subtitle activation function, comprises: When it is detected that the video file does not include subtitle information, displaying a second pop-up window, wherein the second pop-up window includes an eleventh control; In response to the operation of the eleventh control, the automatic subtitle opening function is turned on; The subtitles corresponding to the voice of the video are displayed in the second interface.

10. The method according to any one of claims 6 to 9, characterized in that: The detecting the video file corresponding to the video includes: Detecting subtitle information in the video file; When it is detected that the video file does not include subtitle information, obtaining a volume value when the electronic device plays the video; In the case where it is detected that the video file does not include subtitle information, based on the automatic subtitle opening function, subtitles corresponding to the voice of the video are displayed in the second interface, including: When it is determined that the volume value when the electronic device plays the video is less than a preset volume value, based on the automatic subtitle activation function, subtitles corresponding to the voice of the video are displayed in the second interface.

11. The method according to any one of claims 6 to 9, characterized in that: The detecting the video file corresponding to the video includes: Detecting subtitle information in the video file; When it is detected that the video file does not include subtitle information, obtaining a noise value of a current environment in which the electronic device is located; In the case where it is detected that the video file does not include subtitle information, based on the automatic subtitle opening function, subtitles corresponding to the voice of the video are displayed in the second interface, including: When it is determined that the noise value of the current environment of the electronic device is less than a preset noise value, based on the automatic subtitle activation function, subtitles corresponding to the voice of the video are displayed in the second interface.

12. The method according to any one of claims 6 to 9, characterized in that: The detecting the video file corresponding to the video includes: Detecting subtitle information in the video file; When it is detected that the video file does not include subtitle information, obtaining an audio language corresponding to the video; In the case where it is detected that the video file does not include subtitle information, based on the automatic subtitle opening function, subtitles corresponding to the voice of the video are displayed in the second interface, including: When it is determined that the audio language corresponding to the video is different from the system language of the electronic device, based on the automatic subtitle activation function, subtitles corresponding to the voice of the video are displayed in the second interface.

13. An electronic device, characterized in that: include: one or more processors; one or more memories; The one or more memories store one or more computer programs, and the one or more computer programs include instructions, which, when executed by the one or more processors, enable the electronic device to perform the method as claimed in any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method according to any one of claims 1 to 12.

15. A chip, characterized in that: The chip comprises: A memory for storing instructions; A processor, configured to call and execute the instructions from the memory, so that an electronic device equipped with the chip executes the method as claimed in any one of claims 1 to 12.

Images