Video control method and electronic equipment

Abstract

The invention provides a video control method and electronic equipment, and relates to the technical field of electronic equipment, and the method can achieve the playing control of a video through a voice instruction, and is more convenient for a user to operate. The method is applied to a video playing process of a target video application in the electronic equipment, and comprises the following steps: the electronic equipment receives voice information of a user, and if the electronic equipment can select a target voice instruction corresponding to the voice information from preset voice instructions corresponding to the target video application, the target video application plays the target voice instruction; if yes, determining a corresponding target instruction code based on the target voice instruction; under the condition that the target video application is the current focus application of the electronic equipment, the electronic equipment can judge that the target video application supports instruction processing; and after judging that the target video application supports instruction processing, the electronic equipment can control the video in the target video application based on the target voice instruction corresponding to the target instruction code.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of electronic devices, and in particular to a video control method and electronic device. Background Art

[0002] Currently, when controlling the playback of a video in an electronic device, it is usually necessary to enter the control interface of the video application. The electronic device can only control the playback of the video by receiving the user's touch operation on each control control, and the operation process is relatively cumbersome. Summary of the invention

[0003] The embodiments of the present application provide a video control method and an electronic device, which can realize video playback control through voice commands, making user operation more convenient.

[0004] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0005] In the first aspect, a video control method is provided, which is applied to the process of a target video application in an electronic device playing a video, and the method includes: the electronic device receives a user's voice information, and determines whether the voice information contains a voice instruction, that is, compares it with the voice instruction corresponding to the preset target video application. If the electronic device selects a target voice instruction corresponding to the voice information from the voice instructions corresponding to the preset target video application, the corresponding target instruction code is determined based on the target voice instruction. Then, it is determined whether the target video application is the current focus application of the electronic device. In the case that the target video application is the current focus application, the electronic device determines that the target video application supports instruction processing. The electronic device can control the video in the target video application based on the target voice instruction corresponding to the target instruction code.

[0006] Here, after the determined voice command is mapped and converted into the command code, the corresponding voice command can be transmitted in the electronic device through the command code. The electronic device can recognize the command code, that is, recognize the voice command corresponding to the command code, and control the video in the video application by executing the operation corresponding to the voice command.

[0007] By adopting this technical solution, it is possible to control the playback of the video through instructions in the user's voice information during the video playback process, making the user operation more convenient.

[0008] In a possible implementation of the first aspect, before the electronic device receives the user's voice information, it also includes: when the electronic device detects that the target video application is in a state of playing a video, the interface of the electronic device displays a first prompt message; wherein the first prompt message is used to prompt the user whether to turn on the voice control function. In other words, before the electronic device receives the user's voice information, it is necessary to determine whether the electronic device has turned on the voice control function. When the electronic device detects that the target video application is in a state of playing a video, it can prompt the user whether to turn on the voice control function through a prompt message, that is, the electronic device turns on the voice control function in response to the user's operation, rather than directly turning it on, so that it can choose whether to turn on the voice control function according to the user's needs.

[0009] In a possible implementation of the first aspect, the electronic device receives voice information of the user, including: the electronic device responds to the user's touch operation on the first prompt information, turns on the voice control function, and the electronic device receives the user's voice information. The electronic device can directly respond to the user's touch operation on the first prompt information to turn on the voice control function. Alternatively, the first prompt information includes an opening control, and the opening control is used to turn on the voice control function corresponding to the video application. The electronic device responds to the user's touch operation on the opening control, and turns on the voice control function.

[0010] In a possible implementation of the first aspect, the voice control function supports the target video application, and before the electronic device receives the user's voice information, it also includes: the electronic device responds to the user's activation operation of the control corresponding to the voice control function, and turns on the voice control function. That is, a voice control entrance can be set in the electronic device. If the voice control function is turned on in the entrance, after the video application is opened and the video is played, the electronic device can receive the user's voice information without entering the function activation guide. Among them, the electronic device can be provided with a control corresponding to the voice control function, and the electronic device responds to the user's activation operation of the control to turn on the voice control function. Among them, the preset video application corresponding to the voice control function can be pre-configured in the mobile phone 1. That is to say, after the preset video application plays the video, it enters the preset video scene and can start receiving the user's voice information.

[0011] In a possible implementation of the first aspect, the electronic device turns on the voice control function in response to a user turning on a control corresponding to the voice control function, including: a control corresponding to the voice control function is provided in the electronic device. In the case where the voice control function is configured with a preset video application, the electronic device turns on the voice control function corresponding to the preset video application in response to the turning on operation of the control. The preset video application may be one or more, and in the case where there are multiple preset video applications, it is no longer necessary to select a video application to turn on separately.

[0012] In a possible implementation of the first aspect, the electronic device turns on the voice control function in response to a user's turning on operation on a control corresponding to the voice control function, including: a plurality of voice control function turning on controls corresponding to preset video applications are set in the electronic device, and in response to a user's turning on operation on a certain voice control function turning on control, the voice control function of the corresponding preset video application is turned on.

[0013] In a possible implementation of the first aspect, the electronic device turns on the voice control function in response to a user's turn-on operation on a control corresponding to the voice control function, including: a control corresponding to the voice control function and a turn-on control corresponding to each preset video application are provided in the electronic device. After the electronic device responds to the user's turn-on operation on the control corresponding to the voice control function, the target video application supported by the voice control function is selected, that is, in response to the user's turn-on operation on the turn-on control corresponding to the target video application, the voice control function corresponding to the target video application is turned on.

[0014] In a possible implementation of the first aspect, the electronic device receives the user's voice information, including: when detecting that the target video application is in a state of playing a video, the electronic device receives the user's voice information. That is, if the voice control function of the target video application is enabled in advance at the voice control entrance, when detecting that the target video application is in a state of playing a video, there is no need to display a prompt message indicating that the function is enabled, and voice control can be directly enabled.

[0015] In a possible implementation of the first aspect, an electronic device receives voice information from a user, and if the electronic device selects a target voice instruction corresponding to the voice information from the voice instructions corresponding to a pre-set target video application, then the corresponding target instruction code is determined based on the target voice instruction, including: a voice framework of the electronic device receives the voice information from the user; if the voice framework selects a target voice instruction corresponding to the voice information from the voice instructions corresponding to the pre-set target video application, then the corresponding target instruction code is determined based on the target voice instruction. In an embodiment of the present application, the electronic device receives the user's voice information through a voice framework. The voice framework parses the received voice information, and then maps the parsed voice instructions to event codes.

[0016] In a possible implementation of the first aspect, after determining the corresponding target instruction code based on the target voice instruction, the method further includes: the voice framework sends the target instruction code to the instruction execution framework of the electronic device. The voice framework may send a key value corresponding to the target instruction code to the instruction execution framework.

[0017] In a possible implementation of the first aspect, when the target video application is the current focus application of the electronic device, the electronic device determines that the target video application supports instruction processing, including: the instruction execution framework identifies the current focus application of the electronic device in response to receiving the target instruction code sent by the voice framework, and determines whether the target video application is the current focus application. When the target video application is the current focus application, the instruction execution framework determines that the target video application supports instruction processing. If the identified current focus application is not the target video application, it means that the electronic device has exited the target video application. Then, at this time, the instruction execution framework no longer performs subsequent steps.

[0018] In a possible implementation of the first aspect, an electronic device controls a video in a target video application based on a target voice instruction corresponding to a target instruction code, including: an instruction execution framework constructs a first message, wherein the first message includes a target instruction code; the instruction execution framework sends the first message to the target video application; the target video application obtains the target instruction code in the target message in response to receiving the first message; the target video application performs a corresponding playback control operation on the currently playing video in the target video application based on the target voice instruction corresponding to the target instruction code. Here, in the voice framework of the electronic device, the instruction execution framework and the video application, the instruction codes that can be recognized by the electronic device are transmitted.

[0019] In a possible implementation of the first aspect, an instruction execution framework constructs a first message, including: the instruction execution framework determines whether to bind a target service of a target video application; wherein the target service is used to implement playback control operations for a video in the target video application; if the instruction execution framework does not bind the target service, the instruction execution framework initiates a binding request to the target video application; the target video application returns a proxy object of the target service to the instruction execution framework in response to the received binding request; wherein the proxy object is used to call the target service of the target video application; the instruction execution framework constructs the first message in response to receiving the proxy object; or, the instruction execution framework determines whether to bind the target service of the target video application; if the instruction execution framework binds the target service, the instruction execution framework constructs the first message.

[0020] In a possible implementation of the first aspect, the target video application performs a corresponding broadcast control operation on the currently playing video in the target video application based on the target voice instruction corresponding to the target instruction code, including: the target video application calls the target service based on the target voice instruction corresponding to the target instruction code, and performs the corresponding broadcast control operation on the currently playing video in the target video application. In an embodiment of the present application, the video application can implement specific broadcast control capabilities through a custom service. The electronic device can send a corresponding message to the custom service, and the service implements a specific broadcast control operation according to the message parameters.

[0021] In a possible implementation of the first aspect, after the target video application performs corresponding playback control operations on the currently playing video in the target video application based on the target instruction code, it also includes: the target video application obtains the execution result of the target instruction code, and assembles a second message based on the execution result; the target video application sends the second message to the instruction execution framework; the instruction execution framework responds to the received second message, parses the second message to obtain the execution result; and the instruction execution framework sends the execution result to the voice framework.

[0022] In a possible implementation of the first aspect, after the instruction execution framework sends the execution result to the voice framework, it also includes: the voice framework responds to the received execution result and displays a second prompt information on the interface of the target video application based on the execution result; wherein the second prompt information is used to prompt the user of the execution result corresponding to the target voice instruction. The execution result includes execution success or execution failure. Thus, the prompt information can prompt the user so that the user can know whether his voice has a response.

[0023] In a possible implementation of the first aspect, the method further includes: if the instruction execution framework does not receive the second message within a preset time length, the instruction execution framework unbinds the target service. If the instruction execution framework does not receive the second message within the preset time length, it indicates that the electronic device may have closed the target video application. At this time, unbinding the service of the instruction execution framework can avoid the situation where the binding information of the service is still stored after the application is closed, causing resource redundancy.

[0024] In a second aspect, the present application provides an electronic device, comprising: a communication module, a display screen, a memory and one or more processors; the communication module, the display screen, the memory and the processor are coupled; the memory is used to store computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the electronic device, the electronic device executes the method described in the first aspect above.

[0025] In a third aspect, the present application provides a computer-readable storage medium, in which computer instructions are stored. When the computer-readable storage medium is executed on an electronic device, the electronic device can execute any of the methods described in the first aspect.

[0026] In a fourth aspect, the present application provides a computer program product comprising computer instructions, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect.

[0027] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute a method as described in any one of the methods in the first aspect.

[0028] It can be understood that the electronic device described in the second aspect provided above, the computer-readable storage medium described in the third aspect, the computer program product described in the fourth aspect, and the chip described in the fifth aspect are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a voice control scenario provided in an embodiment of the present application;

[0030] Figure 2 A schematic diagram of the overall architecture of a video control method provided in an embodiment of the present application;

[0031] Figure 3 A schematic diagram of the overall architecture of another video control method provided in an embodiment of the present application;

[0032] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0033] Figure 5 A software structure block diagram of an electronic device provided in an embodiment of the present application;

[0034] Figure 6 A flowchart of a video control method provided in an embodiment of the present application;

[0035] Figure 7 A schematic diagram of a voice-controlled entrance provided in an embodiment of the present application;

[0036] Figure 8 A schematic diagram of another voice-controlled entry provided in an embodiment of the present application;

[0037] Fig. 9 A flowchart of a video control method provided in an embodiment of the present application;

[0038] Fig.10 A schematic diagram of the interface display of a video application of a mobile phone provided in an embodiment of the present application;

[0039] Fig.11 A timing diagram for implementing a video control method provided in an embodiment of the present application;

[0040] Fig.12 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0042] It should be noted that the following terms "first", "second", etc. are only used for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0043] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0044] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0045] At present, it is more and more common to play videos through video applications installed in electronic devices. Generally speaking, electronic devices usually respond to the user's touch operation on multiple control buttons on the video application interface to control the playback of videos. However, the positions of the control buttons on the interface may be scattered and may not be on the same interface, so the user may need to perform multiple operations to achieve the playback control of the video, which is cumbersome. In addition, when the user is far away from the electronic device and the electronic device cannot receive the user's touch operation, the playback control of the video cannot be achieved.

[0046] Therefore, the embodiment of the present application provides a video control method, which can directly transmit voice control instructions to the video application of the electronic device, so that the electronic device performs the operation corresponding to the instruction, and the operation is simpler. Figure 1 In the scenario shown, the electronic device can receive the user's voice command and perform the operation corresponding to the command.

[0047] The overall architecture diagram of a video control method provided in an embodiment of the present application is as follows Figure 2 As shown, after the decision module of the electronic device determines to launch the instruction service, the instruction module of the electronic device can receive the user instruction. Specifically, the decision module is used to detect whether the electronic device is in a preset video scene. Among them, the preset video scene can refer to the scene in which the electronic device is playing a video.

[0048] In some embodiments, the preset video scene may refer to a video obstruction scene, that is, a scene in which the electronic device is not in the user's hand when the electronic device is playing a video. For example, when the decision module detects that the electronic device is playing a video and the electronic device is in a stationary state, it may be determined that the electronic device is in a video obstruction scene.

[0049] The decision module may include only one module or may be composed of a plurality of modules. The embodiment of the present application does not limit the specific implementation of the decision module, but only needs to be able to implement the above functions of the decision module. The instruction module then passes the received instruction to the interaction framework, which then sends the instruction to the corresponding video application, and finally the video application executes the received instruction.

[0050] In some embodiments, the command module may include a voice command module and a gesture command module. Figure 3 As shown, the decision module of the electronic device can determine whether to launch a voice command service or a gesture command service according to the specific video scene. If the voice command service is launched and the voice command module is entered, the electronic device can receive the user's voice command, and then pass the voice command to the interaction framework, and send the voice command to the corresponding application through the interaction framework so that the application executes the corresponding voice command. If the gesture command service is launched and the gesture command module is entered, the electronic device can receive the user's gesture command, and then pass the gesture command to the interaction framework, and send the gesture command to the corresponding application through the interaction framework so that the application executes the corresponding gesture command.

[0051] In the embodiment of the present application, the above-mentioned video control method is specifically introduced by taking voice control as an example.

[0052] The electronic device in the embodiment of the present application may be an electronic device installed with a video application. Exemplarily, the electronic device in the embodiment of the present application may be a tablet computer, a mobile phone, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device, a vehicle-mounted device, and the like. The embodiment of the present application does not impose any special restrictions on the specific form of the electronic device.

[0053] The execution subject of the video control method provided in the embodiment of the present application may be a video control device, and the execution device may be Figure 4 The electronic device shown. At the same time, the execution device can also be a central processing unit (CPU) of the electronic device, or a control module for video control in the electronic device. In the embodiment of the present application, the video control method provided by the embodiment of the present application is described by taking the electronic device executing the video control method as an example.

[0054] The following will describe the implementation of the embodiment of the present application in detail with reference to the accompanying drawings. Taking the above-mentioned electronic device as a mobile phone as an example, the hardware structure of the electronic device (such as the electronic device 400) is introduced. Figure 4 The electronic device 400 shown is merely one example of an electronic device, and the electronic device 400 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 4 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0055] See also Figure 4 , Figure 4 The hardware structure diagram of the electronic device is shown in FIG. Figure 4As shown, the electronic device 400 may include: a processor 410, an external memory interface 420, an internal memory 421, a USB interface 430, a charging management module 440, a power management module 441, a battery 442, an antenna 1, an antenna 2, a mobile communication module 450, a wireless communication module 460, an audio module 470, a speaker 470A, a receiver 470B, a microphone 470C, an earphone interface 470D, a sensor module 480, a button 490, a motor 491, an indicator 492, a camera 493, a display screen 494, and a subscriber identification module (SIM) card interface 495, etc.

[0056] Among them, the above-mentioned sensor module 480 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

[0057] The touch sensor is also called a "touch panel". The touch sensor can be set on the display screen 494, and the touch sensor and the display screen 494 form a touch screen, also called a "touch screen". The touch sensor is used to detect touch operations acting on or near it. The detected touch operation can be transmitted to the application processor to determine the type of touch event and provide corresponding visual output through the display screen 494. In other embodiments, the touch sensor can also be set on the surface of the electronic device 400, which is different from the position of the display screen 494.

[0058] The pressure sensor is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor can be set on the display screen 494. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor, the capacitance between the electrodes changes. The electronic device 400 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 494, the electronic device 400 detects the touch operation intensity according to the pressure sensor. The electronic device 400 can also calculate the touch position according to the detection signal of the pressure sensor 480A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0059] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 400. In other embodiments, the electronic device 400 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0060] The processor 410 may include one or more processing units, for example, the processor 410 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0061] The controller may be the nerve center and command center of the electronic device 400. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0062] The processor 410 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. The memory may store instructions or data that the processor 410 has just used or cyclically used. If the processor 410 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0063] In some embodiments, the processor 410 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0064] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDL) and a serial clock line (SCL). In some embodiments, the processor 410 may include multiple groups of I2C buses. The processor 410 may be coupled to the touch sensor 480K, the charger, the flash, the camera 493, etc. through different I2C bus interfaces. For example: the processor 410 may be coupled to the touch sensor 480K through the I2C interface, so that the processor 410 communicates with the touch sensor 480K through the I2C bus interface to realize the touch function of the electronic device 400.

[0065] The I2S interface can be used for audio communication. In some embodiments, the processor 410 can include multiple groups of I2S buses. The processor 410 can be coupled to the audio module 470 via the I2S bus to achieve communication between the processor 410 and the audio module 470. In some embodiments, the audio module 470 can transmit an audio signal to the wireless communication module 460 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0066] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 470 and the wireless communication module 460 can be coupled via a PCM bus interface. In some embodiments, the audio module 470 can also transmit audio signals to the wireless communication module 460 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication, and the sampling rates of the two interfaces are different.

[0067] The UART interface is a universal serial data bus for asynchronous communication. The bus is a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 410 and the wireless communication module 460. For example, the processor 410 communicates with the Bluetooth module through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 470 can transmit audio signals to the wireless communication module 460 through the UART interface to implement the function of playing music through the Bluetooth headset.

[0068] The MIPI interface can be used to connect the processor 410 with peripheral devices such as the display screen 494 and the camera 493. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 410 and the camera 493 communicate via the CSI interface to realize the shooting function of the electronic device 400. The processor 410 and the display screen 494 communicate via the DSI interface to realize the display function of the electronic device 400.

[0069] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 410 with the camera 493, the display screen 494, the wireless communication module 460, the audio module 470, the sensor module 480, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0070] The USB interface 430 may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 430 may be used to connect a charger to charge the electronic device 400, or may be used to transmit data between the electronic device 400 and a peripheral device. The USB interface 430 may also be used to connect headphones to play audio through the headphones. The USB interface 430 may also be used to connect other electronic devices, such as AR devices, etc.

[0071] It is understandable that the interface connection relationship between the modules shown in this embodiment is only a schematic illustration and does not constitute a structural limitation on the electronic device 400. In other embodiments, the electronic device 400 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0072] The charging management module 440 is used to receive charging input from a charger. In this embodiment of the application, the charger may be a wired charger, and the charging management module 440 may receive charging input from the wired charger through the USB interface 430 (i.e., the above-mentioned charging interface). While the charging management module 440 is charging the battery 442, it may also power the electronic device through the power management module 441.

[0073] The power management module 441 is used to connect the battery 442, the charging management module 440 and the processor 410. The power management module 441 receives input from the battery 442 and / or the charging management module 440, and supplies power to the processor 410, the internal memory 421, the external memory, the display screen 494, the camera 493, and the wireless communication module 460. The power management module 441 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 441 can also be set in the processor 410. In other embodiments, the power management module 441 and the charging management module 440 can also be set in the same device.

[0074] The wireless communication function of the electronic device 400 can be implemented through the antenna 1, the antenna 2, the mobile communication module 450, the wireless communication module 460, the modem processor and the baseband processor.

[0075] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 400 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0076] The RF module can provide a communication processing module for wireless communication solutions including 2G / 3G / 4G / 5G, etc., applied to the electronic device 400. The RF module may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The RF module receives electromagnetic waves from the antenna 1, and performs filtering, amplification, and other processing on the received electromagnetic waves, and transmits them to the modem for demodulation. The RF module can also amplify the signal modulated by the modem, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the RF module may be arranged in the processor 410. In some embodiments, at least some of the functional modules of the RF module may be arranged in the same device as at least some of the modules of the processor 410.

[0077] The mobile communication module 450 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 400. The mobile communication module 450 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 450 can receive electromagnetic waves from the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.

[0078] The mobile communication module 450 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 450 can be set in the processor 410. In some embodiments, at least some functional modules of the mobile communication module 450 can be set in the same device as at least some modules of the processor 410.

[0079] The wireless communication module 460 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 400. For example, in the embodiment of the present application, the electronic device 400 can access the Wi-Fi network through the wireless communication module 460.

[0080] The wireless communication module 460 may be one or more devices integrating at least one communication processing module. The wireless communication module 460 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 410. The wireless communication module 460 may also receive a signal to be sent from the processor 410, modulate the signal, amplify the signal, and convert it into an electromagnetic wave for radiation via the antenna 2.

[0081] In some embodiments, the antenna 1 of the electronic device 400 is coupled to the mobile communication module 450, and the antenna 2 is coupled to the wireless communication module 460, so that the electronic device 400 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0082] The electronic device 400 implements the display function through a GPU, a display screen 494, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 494 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 410 may include one or more GPUs, which execute program instructions to generate or change display information.

[0083] The display screen 494 is used to display images, videos, etc. The display screen 494 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 400 may include 1 or N display screens 494, where N is a positive integer greater than 1.

[0084] In the embodiment of the present application, the display screen 494 can be used to display the interface of the electronic device. The display screen 494 can be used to display the system desktop of the electronic device, and the system desktop of the electronic device can include multiple application icons. The application icons can be displayed at different positions of the display screen 494, that is, they constitute the layout of the system desktop of the electronic device.

[0085] In the embodiment of the present application, the display screen 494 may be used to display a display schematic diagram of an application interface of the electronic device.

[0086] The electronic device 400 can realize the shooting function through ISP, camera 493, video codec, GPU, display screen 494 and application processor.

[0087] ISP is used to process the data fed back by camera 493. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 493.

[0088] The camera 493 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 400 may include 1 or N cameras 493, where N is a positive integer greater than 1.

[0089] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 400 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0090] Video codecs are used to compress or decompress digital videos. The electronic device 400 may support one or more video codecs. Thus, the electronic device 400 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0091] NPU is a neural network (NN) computing processor, which can quickly process input information and continuously self-learn by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain. NPU can realize applications such as intelligent cognition of electronic device 400, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0092] The external memory interface 420 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 400. The external memory card communicates with the processor 410 through the external memory interface 420 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0093] The internal memory 421 may be used to store computer executable program codes, which include instructions. The processor 410 executes various functional applications and data processing of the electronic device 400 by running the instructions stored in the internal memory 421. For example, in an embodiment of the present application, the processor 410 may execute the instructions stored in the internal memory 421, and the internal memory 421 may include a program storage area and a data storage area.

[0094] The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 400 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 421 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0095] The electronic device 400 can implement audio functions such as music playing and recording through the audio module 470, the speaker 470A, the receiver 470B, the microphone 470C, the headphone jack 470D, and the application processor.

[0096] The audio module 470 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 470 can also be used to encode and decode audio signals. In some embodiments, the audio module 470 can be arranged in the processor 410, or some functional modules of the audio module 470 can be arranged in the processor 410.

[0097] The speaker 470A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 400 can listen to music or listen to a hands-free call through the speaker 470A.

[0098] The receiver 470B, also called a "earpiece", is used to convert an audio electrical signal into a sound signal. When the electronic device 400 receives a call or a voice message, the voice can be received by placing the receiver 470B close to the human ear.

[0099] Microphone 470C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 470C to input the sound signal into the microphone 470C. The electronic device 400 can be provided with at least one microphone 470C. In some embodiments, the electronic device 400 can be provided with two microphones 470C, which can not only collect sound signals but also realize noise reduction function. In some embodiments, the electronic device 400 can also be provided with three, four or more microphones 470C to realize the collection of sound signals, noise reduction, identification of sound sources, realization of directional recording function, etc. In the embodiment of the present application, after the voice control function is turned on, in the supported video application, the electronic device can receive the user's voice information through the microphone.

[0100] The earphone interface 470D is used to connect a wired earphone and can be a USB interface 430 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0101] The key 490 includes a power key, a volume key, etc. The key 490 may be a mechanical key or a touch key. The electronic device 400 receives the key 490 input and generates a key signal input related to the user settings and function control of the electronic device 400.

[0102] Motor 491 can generate vibration prompts. Motor 491 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. Touch operations acting on different areas of the display screen 494 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0103] Indicator 492 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0104] The SIM card interface 495 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 400 by inserting the SIM card interface 495 or pulling the SIM card interface 495 out. The electronic device 400 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 495 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0105] The methods in the following embodiments can all be implemented in an electronic device 400 having the above-mentioned hardware structure. Multiple cards can be inserted into the same SIM card interface 495 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 495 can also be compatible with different types of SIM cards. The SIM card interface 495 can also be compatible with external memory cards. The electronic device 400 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 400 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 400 and cannot be separated from the electronic device 400.

[0106] although Figure 4 Not shown, the electronic device 400 may also include a flash, a micro-projection device, a near field communication (NFC) device, etc., which will not be described in detail here.

[0107] After introducing the hardware structure of the electronic device, the system architecture of the electronic device provided by the present application is introduced. The system architecture of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes a system with a layered architecture as an example to exemplify the software structure of the electronic device.

[0108] Figure 5 4 is a software structure diagram of the electronic device 400 of an embodiment of the present invention. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

[0109] The application layer can include a series of application packages.

[0110] like Figure 5 As shown, the application package may include applications such as camera, video, map, WLAN, music, short message, gallery, call, navigation, Bluetooth, smart voice, etc.

[0111] The application package may also include an intelligent assistant application, and the intelligent assistant application may include intelligent voice, intelligent gesture, etc. The intelligent voice may receive voice information input by the user, and the intelligent gesture may receive gesture information input by the user.

[0112] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0113] like Figure 5 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, an instruction execution framework, and the like.

[0114] The command execution framework is used to receive voice commands sent by intelligent voice or gesture commands sent by intelligent gesture, and pass the received voice commands or gesture commands to the video application.

[0115] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0116] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0117] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0118] The phone manager is used to provide communication functions of the electronic device 400. For example, the management of call status (including connecting, hanging up, etc.).

[0119] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0120] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0121] Android runtime includes core library and virtual machine. Android runtime is responsible for scheduling and management of Android system. Core library consists of two parts: one is the function that Java language needs to call, and the other is Android core library.

[0122] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0123] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as openGL ES), 2D graphics engine (such as SGL), etc.

[0124] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple common audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0125] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0126] In the embodiment of the present application, when the touch sensor receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the input event from the kernel layer and identifies the control corresponding to the input event.

[0127] The methods in the following embodiments can all be implemented in an electronic device having the above hardware structure and the above software architecture.

[0128] In the embodiment of the present application, the electronic device is a mobile phone 1 as an example, the mobile phone 1 can transmit instructions to a third-party video application and return a response instruction execution result. For example, the mobile phone 1 can receive a user's voice command to control the video application.

[0129] A flow chart of a video control method provided in an embodiment of the present application is as follows: Figure 6 As shown, during the process of video application 1 of mobile phone 1 playing a video, after voice control is started, mobile phone 1 can receive the user's voice information. First, determine whether the voice information contains a voice instruction, that is, compare it with the voice instruction pre-set in mobile phone 1. If the voice information does not contain a voice instruction, the process ends. If the voice information contains a voice instruction, select the corresponding target voice instruction from it and determine the corresponding target instruction code. Then, determine whether video application 1 is the current focus application, that is, determine whether video application 1 has not exited and is still playing the video. If video application 1 is not the current focus application, the process ends. If video application 1 is the current focus application, it can be determined that video application 1 supports instruction processing, otherwise the above process cannot be triggered during the process of video application 1 playing the video. At this time, mobile phone 1 can control the video being played by video application 1 according to the voice instruction corresponding to the target instruction code. As a result, mobile phone 1 realizes the control of video by voice instruction during the process of playing the video.

[0130] Here, after the determined voice command is mapped and converted into the command code, the corresponding voice command can be transmitted in the electronic device through the command code. The electronic device can recognize the command code, that is, recognize the voice command corresponding to the command code, and control the video in the video application by executing the operation corresponding to the voice command.

[0131] It is understandable that before the mobile phone 1 can receive the user's voice information, the mobile phone 1 has already started the voice control function.

[0132] In an embodiment of the present application, a voice control entrance may be provided in the mobile phone 1. For example, a setting option corresponding to the voice control entrance is included in the setting interface of the mobile phone 1. In response to the user's touch operation on the setting option, the voice control entrance is entered. If the voice control function is turned on in the voice control entrance, the electronic device can directly control the video playback through the received user's voice command after opening the video application and playing the video in response to the user's operation. If the voice control function is not turned on in the voice control entrance, after the electronic device opens the video application and plays the video in response to the user's operation, the application interface may pop up a prompt message of the function activation guide to prompt the user whether to turn on the voice control function.

[0133] For example, a start control corresponding to the voice control function may be provided in the mobile phone 1, and the mobile phone 1 turns on the voice control function in response to the user's turn-on operation of the start control. Among them, a preset video application corresponding to the voice control function may be pre-configured in the mobile phone 1. That is, after the preset video application plays a video, it enters the preset video scene and can start receiving the user's voice information.

[0134] Specifically, only one control corresponding to the voice control function may be provided in mobile phone 1, and the voice control function is pre-configured with a supported preset video application in mobile phone 1. In response to the activation operation of the control, mobile phone 1 activates the voice control function corresponding to the preset video application. The preset video application may be one or more. In the case of multiple preset video applications, there is no need to select a specific video application to be activated separately. For example, the voice control function in mobile phone 1 supports application 1 and application 2. After mobile phone 1 responds to the activation operation of the control corresponding to the voice control function by the user, both application 1 and application 2 activate the voice control function.

[0135] Alternatively, multiple voice control function activation controls corresponding to preset video applications are set in mobile phone 1, and in response to the user's activation operation on a certain voice control function activation control, the voice control function of the corresponding preset video application is activated. For example, the voice control function in mobile phone 1 supports application 1 and application 2, application 1 corresponds to voice control function activation control 1, and application 2 corresponds to voice control function activation control 2. In response to the user's activation operation on voice control function activation control 1, mobile phone 1 activates the voice control function corresponding to application 1. After application 1 plays the video, voice control is activated, and the user's voice information can be received. If the voice control function corresponding to application 2 is not activated, voice control will not be activated immediately after detecting that application 1 plays the video.

[0136] Alternatively, controls corresponding to the voice control function and start controls corresponding to each preset video application are provided in mobile phone 1. After mobile phone 1 responds to the user's start operation on the control corresponding to the voice control function, it is also necessary to select the target video application supported by the voice control function. For example, the preset video applications that can be supported by the voice control function in mobile phone 1 include application 1, application 2, and application 3. Application 1 corresponds to start control 1, application 2 corresponds to start control 2, and application 3 corresponds to start control 3. After mobile phone 1 responds to the user's start operation on the control corresponding to the voice control function, in response to the user's start operation on start control 1, mobile phone 1 starts the voice control function corresponding to application 1. After playing the video in application 1, the preset video scene is entered, voice control is started, and mobile phone 1 can receive the user's voice information.

[0137] like Figure 7 As shown, Figure 7 A schematic diagram of a voice control entry provided in an embodiment of the present application. An option control corresponding to the voice control function may be provided in the setting interface of the mobile phone 1, and the mobile phone 1 displays the first interface 701 of the voice control in response to the user's touch operation on the option control. In the first interface 701, a start control 702 for the voice control function may be included. In response to the user's touch operation on the start control 702, the electronic device turns on the voice control function. That is, after the electronic device opens the preset video application supported by the voice control function and plays the video, the voice control function is started, and the playback of the video being played can be controlled in response to the received voice command of the user.

[0138] In some embodiments, Figure 8 As shown, Figure 8 Another schematic diagram of a voice control entry provided in an embodiment of the present application. An option control corresponding to the voice control function may be provided in the setting interface of the mobile phone 1. In response to the user's touch operation on the option control, the mobile phone 1 displays the first interface 801 of the voice control. In the first interface 801, a voice control function start control 802 may be included. In response to the user's touch operation on the start control 802, the electronic device starts the voice control function.

[0139] The first interface 801 may also include an option 803 corresponding to scenes and applications supported by the voice control function. In response to the user's touch operation on the option 803, the electronic device may display a second interface 804. Scenes and applications supported by the voice control function are displayed in the second interface 804. Figure 8As shown, the supported scenes may include video scene 805 or alarm scene. The video control method of the embodiment of the present application takes the supported scene as video scene 805 as an example, and multiple video applications installed in mobile phone 1 are displayed under video scene 805. Mobile phone 1 can determine the target video application supported by the voice control function in response to the user's touch operation on the corresponding control of the target video application in multiple video applications. In other words, the user can manually manage the scenes and applications. After turning on the corresponding control of the target video application, the target video application supports scene-based opening, that is, when the target video application detects that it is in a preset video scene, the voice control function can be automatically turned on. After closing the corresponding control of the target video application, the target video application no longer supports scene-based opening. Among them, the preset video scene can refer to the scene in which the target video application is playing a video.

[0140] In some embodiments, the preset video scene may refer to a video obstacle scene, that is, a scene in which the video application is playing a video and the mobile phone 1 is not in the user's hand. For example, it is detected that the mobile phone 1 is on a stand, or it is detected that the mobile phone 1 is in a stationary state.

[0141] For example, mobile phone 1 responds to the user's touch operation on the corresponding control of application 1 and determines that application 1 is the target video application. After application 1 is opened in mobile phone 1, if application 1 starts playing the video in response to the user's operation, it is determined that the preset video scene has been entered, and it can be considered that voice control has been started. Mobile phone 1 can directly respond to the voice command in the received user's voice information to control the playback of the video in application 1.

[0142] If the activation control corresponding to application 1 is not turned on in the displayed application list, the voice control will not be turned on directly after application 1 is opened in mobile phone 1 and the video starts to play. Mobile phone 1 cannot directly respond to the voice command received from the user to control the playback of the video in application 1. At this time, a prompt message for the function activation guide may be displayed in the interface of application 1, and mobile phone 1 responds to the user's operation in the prompt message to activate the voice control function corresponding to application 1. Among them, the displayed prompt message may include an activation control, and mobile phone 1 responds to the user's touch operation on the activation control to activate the voice control function corresponding to application 1. After the voice control function is turned on, mobile phone 1 can control the video being played in application 1 through the voice command received from the user. Among them, before the prompt message is displayed in the interface of application 1, application 1 is playing a video.

[0143] The embodiment of the present application will automatically turn on voice control (for example, display a prompt message that voice control has been started) or turn on voice control according to the function start guide when it is detected that application 1 is in the state of playing a video, that is, in a preset video scene, which can save device power consumption. It is understandable that before automatically turning on voice control or displaying the function start guide, application 1 has not exited or exited the video playback. In other words, if application 1 exits or exits the video playback state before automatically turning on voice control or displaying the function start guide, voice control will not be automatically turned on or the function start guide will not be displayed.

[0144] A video control method provided in an embodiment of the present application is introduced in detail below.

[0145] See also Fig. 9 , Fig. 9 A flow chart of a video control method provided in an embodiment of the present application. It is assumed that the user has not enabled the voice control function of the target video application in advance at the voice control entrance. Fig. 9 As shown, after the target video application is turned on and the video starts playing, the electronic device can enter the function start-up guide, that is, the prompt information of the function start-up guide is displayed in the application interface. The electronic device can respond to the user's touch operation on the prompt information to turn on the voice control function corresponding to the video application. For example, the user's single-click operation is set to indicate turning on, and the electronic device responds to the user's single-click operation on the prompt information to turn on the voice control function corresponding to the video application. It can also be that the electronic device displays the start control in response to the user's pressing operation on the prompt information. The electronic device responds to the user's touch operation on the start control to turn on the voice control function corresponding to the video application. Among them, the touch operation may include single-click, double-click, press, slide, etc., which are not limited in the embodiments of the present application.

[0146] The prompt information may also include an opening control, which is used to open the voice control function corresponding to the target video application. For example, the prompt information may be "Smart Assisted Interaction: Automatically open voice control during video playback", and the prompt information may also include a status bar icon corresponding to the voice control function. Figure 8 The start control here corresponds to the start control of the voice control function in the voice control entrance (supporting the target video application), and the operation of the start control is equivalent to the operation of the start control of the voice control function in the voice control entrance.

[0147] The prompt information may also include a cancel control. In response to the user's touch operation on the cancel control, the electronic device may cancel the display of the prompt information, which means that the voice control function corresponding to the target video application is not enabled at this time.

[0148] If the electronic device responds to the user's touch operation on the opening control, the entry status prompt is displayed. That is, the entry status prompt information is displayed on the application interface. For example, the prompt information may be "Voice control has been started", wherein the prompt information also includes a status bar icon. Among them, a cancel control is also provided at the prompt information, and the electronic device can cancel the display of the prompt information in response to the user's touch operation on the cancel control. At this time, the video application has successfully turned on voice control, and the video can be played and controlled by the user's voice command.

[0149] After displaying a prompt message for a period of time (for example, 1 second, 2 seconds, etc.), or canceling the display of the prompt message in response to a user operation, the electronic device can enter the operation guide. That is, the prompt message of the operation guide is displayed in the application interface. For example, the prompt message may be "Try to say "fast forward"", wherein the prompt message also includes a status bar icon. A skip control is also provided at the prompt message, and the electronic device can skip displaying the prompt message of the operation guide in response to the user's touch operation on the skip control. A number limit (such as 2 or 3 times, etc.) can be set for the operation guide. For example, after entering a video application and starting voice control for the first time, the prompt message corresponding to the operation guide can be displayed three times in succession. In the subsequent voice control process, if the video application is not completely closed, the prompt message of the operation guide may no longer be displayed after starting voice control again in the video application.

[0150] Among them, if the electronic device receives the user's touch operation to skip the control, it can skip only the operation guidance this time, or it can directly skip all operation guidance and directly enter the formal voice control.

[0151] At this point, the user can operate according to the operation guidance, such as trying to say voice commands such as "fast forward" or "pause". The electronic device responds to the received voice command and executes the voice command, and can display feedback information of the execution result in real time on the application interface. For example, if the voice command tried is "fast forward", the electronic device receives the voice command and executes it, and the video is fast forwarded. The information "Fast forward successful" can be displayed on the application interface, or it can also be displayed as "Fast forward X seconds", etc. This application does not limit the specific content and form of the feedback information displayed.

[0152] In some embodiments, the feedback information of the execution result displayed in real time on the application interface may also refer to the control state of the video after the video executes the voice command. For example, if the voice command attempted to be said is "pause", the electronic device receives the voice command and executes it, and the control state of the video is that the video playback is paused, and there is no need to display prompt information such as "pause successful" on the application interface. In other words, the user can directly judge whether the voice command is successfully executed based on the control state of the current video. For example, if the video playback is paused, it means that the voice command is successfully executed.

[0153] If the video fails to execute the voice command, a prompt message such as "pause failed" may be displayed on the application interface to prompt the user that the voice command execution failed.

[0154] After the operation guide is completed or the operation guide is skipped, the electronic device enters the formal voice control. The electronic device responds to the received voice command, executes the voice command, and can display the feedback information of the execution result in real time on the application interface. For example, if the voice command received by the electronic device is "pause", the electronic device receives the voice command and executes it, the video playback is paused, and the information "pause successful" can be displayed on the application interface.

[0155] After the electronic device exits the video application, the interface of the electronic device displays an exit status prompt, that is, a notification message of the exit status. For example, the notification message may be "Smart Assisted Interaction Has Exited", wherein the notification message also includes a status bar icon. The electronic device may display the notification message within a preset duration. At this time, it means that the video application of the electronic device has exited the smart assisted interaction, that is, exited the voice control function. Among them, the above-mentioned electronic device exits the video application, including the electronic device closing the video application, the video application running in the background, and the video application becoming a non-focus application.

[0156] In some embodiments, when the electronic device receives a voice command corresponding to "exit application", the interface of the electronic device displays the above-mentioned exit status prompt.

[0157] like Fig.10 As shown, Fig.10 A schematic diagram of the interface display of a video application of a mobile phone provided in an embodiment of the present application, wherein it is assumed that the user has not enabled the voice control function in advance at the voice control entrance of the mobile phone. Fig.10In the example, after the mobile phone opens the video application, it enters the function activation guide. That is, after the mobile phone opens the video application and starts playing the video, the function activation guide prompt information 902 is displayed on the application interface 901, and the prompt information includes an activation control 906, which is used to immediately activate the voice control function corresponding to the video application. For example, the prompt information 902 can be "Smart Assisted Interaction: Automatically activate voice control during video", and the prompt information 902 can also include a status bar icon 903 corresponding to the voice control function.

[0158] Understandably, Fig.10 The display form (capsule shape) of the prompt information 902 is only an example, and other display forms are also possible. Fig.10 The display content of the prompt information 902 is also just an example, and it can also be other display content, as long as it can express the corresponding meaning of the function activation guidance. Fig.10 The shape of the status bar icon 903 is also only an example, and may be other shapes. The embodiment of the present application does not limit the display form, display content, display position of the prompt information 902, and the shape of the status bar icon 903.

[0159] Among them, the mode supported by the current scene can be displayed at the status bar 904 of the mobile phone, such as Fig.10 , a microphone icon 905 is displayed. That is, the microphone icon indicates that a microphone is needed in the current scene, that is, the current video application supports the voice control function, and the voice control function requires the use of a microphone. In some embodiments, the modalities supported by the current scene may also be gesture control, eye control, etc. For example, if the current video application supports the gesture control function, the gesture control icon is displayed in the status bar of the mobile phone. If the current video application supports the eye control function, the eye control icon is displayed in the status bar of the mobile phone.

[0160] After the mobile phone responds to the user's touch operation on the start control 906, a prompt message 907 is displayed on the application interface 901 to prompt that the voice control has been started. For example, the prompt message 907 can be "Voice control has been started". Among them, the prompt message 907 also includes a status bar icon for voice control. Among them, a cancel control 908 is also provided at the prompt message 907, and the mobile phone can cancel the display of the prompt message 907 in response to the user's touch operation on the cancel control 908. And, at the status bar 904 of the application interface 901, the status bar icon and the microphone icon can be displayed respectively. Among them, the status bar icon is displayed at the status bar of the mobile phone application interface, and it can be intuitively known that the mobile phone is in voice control at this time. If the status bar icon is not displayed at the status bar of the mobile phone application interface, it means that the voice control of the mobile phone has not been started, or the voice control has been exited.

[0161] After displaying the prompt information 907 for a period of time, or canceling the display of the prompt information 907 in response to a user operation, the mobile phone can enter the operation guide. The mobile phone displays the prompt information 909 of the operation guide on the application interface 901. Among them, the prompt information 909 also includes a voice control status bar icon and a skip control 910. In response to the user's touch operation on the skip control 910, the mobile phone can skip displaying the prompt information 909 of the operation guide. In addition, the status bar information of the application interface 901 also displays the status bar icon and the microphone icon.

[0162] At this time, the user can operate according to the operation guidance, such as trying to say voice commands such as "fast forward" or "pause". The mobile phone responds to the received voice command, executes the voice command, and can display feedback information of the execution result in real time on the application interface 901. For example, if the voice command tried to be said is "fast forward", then after the mobile phone receives and executes the voice command, the information "Fast forward successful" is displayed on the application interface, or it can also display "Fast forward X seconds" and so on. This application does not limit the specific content and form of the prompt information.

[0163] After the operation guide is completed or the operation guide is skipped, the mobile phone enters the formal voice control. The mobile phone responds to the received voice command, executes the voice command, and displays the feedback information of the execution result in real time on the application interface of the mobile phone.

[0164] After the mobile phone exits the video application, or after the mobile phone receives the voice command corresponding to "exit application", the mobile phone application interface 901 displays the exit status prompt, that is, the exit status notification information 911. At this time, the status bar information of the application interface 901 no longer displays the status bar icon and the microphone icon.

[0165] In some embodiments, if the user turns on the voice control function in advance at the voice control entrance of the mobile phone, then after the mobile phone opens the video application, an entry status prompt message 907 is directly displayed on the application interface 901 to prompt that the voice control has been started.

[0166] See also Fig.11 , Fig.11 This is a timing diagram for implementing a video control method provided in an embodiment of the present application. Fig.11 As shown, the voice framework of the electronic device receives the user's voice information, determines the voice command in the voice information, and maps the voice command to an event code (i.e., the above-mentioned command code). Among them, the voice command may include play, pause, fast forward, fast rewind, previous, next, replay, like, favorite, follow, double speed, etc.

[0167] Specifically, the received voice information is compared with the pre-stored voice instructions. If the voice information contains a certain voice instruction, the event code of the voice instruction can be mapped. Among them, different video applications have different functions, and different video applications can correspond to different voice instructions. Since the voice control is started after the target video application is opened and the video starts to play, the voice instructions corresponding to the target video application pre-stored in the electronic device can be determined, and the received voice information is compared with the voice instructions corresponding to the target video application. For example, after opening application 1 and playing the video, voice control is started, and the voice framework receives the user's voice information. The voice framework compares the voice information with the pre-stored voice instruction set 1 corresponding to application 1 to determine whether there are voice instructions in voice instruction set 1 in the voice information.

[0168] If the voice message does not contain any voice command, no mapping is performed, or the mapping fails. For example, if the voice message received by the voice framework is "play", then it can be determined that the voice command corresponding to the voice message is "play". For another example, if the voice message received by the voice framework is "play video", then it can be determined that the voice command corresponding to the voice message is also "play".

[0169] That is to say, after voice control is started, the voice framework can receive a lot of voice information from the user, but not every voice information received contains voice commands. The voice framework will parse the received voice information and then map the parsed voice commands to event codes. It can be understood that the voice framework is an application package that can receive voice information input from the user.

[0170] In some embodiments, the text corresponding to the voice instruction contained in the received voice information is not necessarily exactly the same as the text corresponding to the pre-stored voice instruction. For example, the voice information received by the voice framework is "stop video", and the corresponding voice instruction can also be considered to be "pause". For another example, the voice information received by the voice framework is "replay", and the corresponding voice instruction can be considered to be "replay". In other words, when the voice framework parses the user's voice information, it can also perform semantic analysis. If there is a text in the voice information that has the same or similar semantics as the text corresponding to a stored voice instruction, it can also be considered that the voice information contains the corresponding voice instruction.

[0171] In the embodiment of the present application, if the corresponding voice instruction is determined according to the received voice information, the voice instruction is pre-set with support parameters, but the parameters currently determined according to the voice information exceed the support range of the video application. Then, the voice framework can also determine the corresponding event code according to the voice instruction, send it to the instruction execution framework, and then send it to the video application by the instruction execution framework to execute the voice instruction, but the video application still executes the video application according to the pre-set support parameters. Or, the voice framework cannot determine the corresponding event code according to the voice instruction. For example, the received voice information is "100 times the speed of play", and the voice framework can determine that the corresponding voice instruction is "speed", and the parameter is 100 times. Since the pre-set support parameters of the voice instruction include 2 times and 3 times the speed, the default is 2 times, that is, the video application only supports 2 times or 3 times the speed of playing video. Therefore, the currently determined instruction parameters are far beyond the support range of the video application. In this case, the voice instruction can be set to be executed at the default 2 times speed, or it can be set to be executed at the maximum multiple of 3 times in the support parameters. Or, in this case, the voice framework does not perform subsequent steps and cannot execute the voice instruction.

[0172] In an embodiment of the present application, voice instructions and corresponding event codes may be pre-stored in an electronic device. The information may be stored in a list, for example, as shown in Table 1 below. As can be seen from Table 1, the list stores event codes and corresponding descriptions and supporting parameters. The description indicates the voice instruction corresponding to the event code.

[0173] Table 1

[0174]

[0175] In Table 1, the event code corresponding to the voice command "play" is EVENT_CODE_MEDIA_PLAY, and the key value corresponding to the event code is 126. That is to say, the key value corresponding to the voice command is what the voice framework finally sends to the command execution framework. The event code corresponding to the voice command "pause" is EVENT_CODE_MEDIA_PAUSE, and the key value corresponding to the event code is 127. The event code corresponding to the voice command "fast forward" is EVENT_CODE_MEDIA_STEP_FORWARD, and the key value corresponding to the event code is 274. The supported parameter is the unit of fast forward time in seconds (s), and the default is 5s. The event code corresponding to the voice command "fast rewind" is EVENT_CODE_MEDIA_STEP_BACKWORD, and the key value corresponding to the event code is 275. The supported parameter is the unit of fast rewind time in seconds (s), and the default is 5s. The event code corresponding to the voice command "next" is EVENT_CODE_MEDIA_NEXT, and the key value corresponding to the event code is 87. The event code corresponding to the voice command "previous" is EVENT_CODE_MEDIA_PREVIOUS, and the key value corresponding to the event code is 88. The event code corresponding to the voice command "replay" is EVENT_CODE_MEDIA_REPLAY, and the key value corresponding to the event code is 800. The event code corresponding to the voice command "like" is EVENT_CODE_MEDIA_LIKE, and the key value corresponding to the event code is 801. The event code corresponding to the voice command "favorite" is EVENT_CODE_MEDIA_STAR, and the key value corresponding to the event code is 802. The event code corresponding to the voice command "follow" is EVENT_CODE_MEDIA_FOLLOW, and the key value corresponding to the event code is 803. The event code corresponding to the voice command "speed" is EVENT_CODE_MEDIA_FAST_FORWARD, and the key value corresponding to the event code is 90, and the supported parameter is the default 2x speed playback. The embodiment of the present application can add or delete the stored event codes, and there is no restriction on this.

[0176] Among them, other undefined event codes EVENT_CODE_MEDIA_OTHERS are also stored in Table 1, and the corresponding key value is 112. That is to say, it means that a position can be reserved by setting the event code, and other newly added voice commands can be set at this position. For example, the event codes corresponding to voice commands such as adjusting clarity, adjusting brightness, and adjusting sound can be set and stored in the above list. It can be understood that Table 1 is only an example, and there can be multiple undefined event codes.

[0177] Among them, the support parameters can be set according to the capabilities of the video application. For example, the video application can support 2x, 3x and 4x speed playback, then the support parameters corresponding to the voice command "speed" may include 2x, 3x and 4x speed. Among them, the default 2x speed playback means that if the determined voice command does not contain parameters, the voice command is executed at 2x speed by default. For example, the received voice information is "play at double speed", and the corresponding voice instruction is determined to be "double speed", then when the video application plays at double speed, the default is 2x speed. If the received voice information is "play at 3x speed", the corresponding voice instruction is determined to be "double speed", and the corresponding parameter is 3x speed, then the video application plays the video at 3x speed.

[0178] After the voice framework maps the received voice command to the corresponding event code, the key value corresponding to the event code is sent to the command execution framework through the interface of injectVideoEvent.

[0179] After receiving the key value corresponding to the event code, the instruction execution framework first determines whether the currently identified focus application supports instruction processing. Among them, the focus application refers to the application that is currently processing the event, and whether the target video application has been exited can be determined by identifying whether the current focus application is still the target video application. It can be understood that the electronic device controls the playback of the video in the target video application through the received user's voice instructions after turning on the video application (set as the target video application) and playing the video. Among them, after turning on the target video application and playing the video, the target video application can receive various operations from the user, and the focus application at this time is the target video application. Therefore, if the instruction execution framework identifies that the current focus application is still the target video application after receiving the key value corresponding to the event code, it can be determined that the focus application (i.e., the target video application) supports instruction processing. Otherwise, voice control will not be started after the target video application is turned on.

[0180] If the identified current focus application is not the target video application, it means that the electronic device has exited the target video application. Then, the voice command received after the target video application is opened and the video is played cannot control the playback of the video in the target video application.

[0181] Among them, if the current focus application identified is not the target video application, there may be two situations. The first situation is that the applications supported by the voice control function set in the voice control entrance include application 1 (target video application) and application 2. It can be understood that application 1 and application 2 here support the voice control function, that is, support instruction processing, wherein application 1 supports instruction processing for voice instructions of application 1, and application 2 supports instruction processing for voice instructions of application 2. Assume that after application 1 is turned on and the video is played, the voice framework receives the user's voice information, maps the corresponding voice instructions in the voice information to event codes and sends them to the instruction execution framework, and the instruction execution framework recognizes that the current focus application is application 2. At this time, the instruction execution framework can determine that application 2 supports the voice control function, that is, supports instruction processing. However, since the voice framework receives the user's voice information after opening application 1, that is, the user's intention recognized by the electronic device at this time is to control the playback of the video in application 1, the voice instruction determined by the voice framework based on the user's voice information is a voice instruction for application 1. Therefore, if the instruction execution framework recognizes that the current focus application is application 2, and application 2 cannot support instruction processing for the voice instruction of application 1, it can be determined that the current focus application does not support instruction processing.

[0182] The second case is that the applications supported by the voice control function set in the voice control entry do not include application 2. Assume that after opening application 1 and playing the video, the voice framework receives the user's voice information, maps the corresponding voice command in the voice information to an event code and sends it to the command execution framework, and the command execution framework recognizes that the current focus application is application 2. At this time, the command execution framework can determine that application 2 does not support the voice control function, that is, it does not support command processing.

[0183] For example, if the voice control function is turned on after the application 1 is turned on and plays a video, the voice framework can receive the user's voice information. In response to the user's operation, the mobile phone 1 splits the screen, and the application 1 plays the video in the split screen window 1, and the split screen window 2 displays the application 2. If the mobile phone 1 responds to the user's operation on the application 2 in the split screen window 2, at this time, it can be determined that the current focus application is application 2, and the application 1 cannot be voice controlled, that is, the voice command received by the voice framework is discarded, and the subsequent steps will not be performed.

[0184] The focus application may be determined by an existing method for determining the focus application, for example, by calling the activityManager.getRunningTasks(1).get(0).topActivity method to determine the current focus application.

[0185] In some embodiments, after identifying the focus application, it can also be detected whether it has exited the static state, that is, whether the electronic device is still in a preset video scene. If the electronic device is not in the preset video scene, that is, the preset voice control condition is not met, the electronic device can directly exit the voice control function.

[0186] In some embodiments, if the electronic device does not meet the preset voice control conditions, a prompt message may be displayed in the application to prompt the user whether to continue using the voice control function. If the user chooses to continue using it, the electronic device may also respond to the user's touch operation on the corresponding control and continue to execute subsequent steps, that is, the step of the instruction execution framework sending a broadcast control message corresponding to the voice instruction to the application.

[0187] If the focus application supports command processing, it will determine whether the voice execution framework is bound to the focus application. If the voice execution framework is already bound to the focus application, the voice execution framework can call the application's service to perform the operation corresponding to the command. If the command execution framework is not yet bound to the focus application, it will bind the corresponding service (BindService).

[0188] Specifically, the voice execution framework can directly send a request for binding the service to the target video application. After receiving the request, the target video application will return the binding result to the voice execution framework. If the target video application returns a null value, it means that the binding has failed, that is, the user may have closed the target video application or exited the video playback at this time, then the voice control function has been exited, and there is no need to continue with the subsequent steps. If the binding is successful, the target video application will return the proxy object corresponding to the service (Service) to the instruction execution framework, and the proxy object can be used to call the service of the target video application. The target video application can implement specific broadcast control capabilities through the above services, including play / pause, fast forward / rewind, rate adjustment, previous / next (short video) and like / favorite / follow operations.

[0189] In an embodiment of the present application, the instruction execution framework needs to send a request for binding the service to the target video application only after receiving the user's voice instruction for the first time. If the voice execution framework has received the user's voice instruction and the electronic device has not closed the target video application, then it can be determined that the voice execution framework has been bound to the service of the target video application, and the voice instruction framework has obtained the proxy object corresponding to the service. In other words, after the target video application is opened for the first time, or after the target video application is reopened after exiting the application, the voice execution framework will bind the service. The service will be unbound only after exiting the application. In this way, multiple requests for binding services before the application exits can be avoided, resulting in a waste of power consumption. It can also be avoided that the binding information of the service is still saved after the application is closed, resulting in resource redundancy.

[0190] In the embodiment of the present application, the video playback control event will rely on three-party docking, and different manufacturers may have different docking types, such as different video applications. Among them, information synchronization can be performed through messages. That is to say, after the service is successfully bound, the instruction execution framework can splice the message object according to the target key value of the received event code.

[0191] After the message splicing is successful, a broadcast control message is sent to the target video application, which contains the target key value. After receiving the broadcast control message, the application parses the message to obtain the target key value. The target video application determines the corresponding voice instruction according to the event code corresponding to the target key value, and calls the service according to the voice instruction to implement the corresponding broadcast control operation, that is, executes the voice instruction.

[0192] After the execution of the target video application's instructions is completed, the execution result is packaged into a result message and returned to the instruction execution framework. After receiving the result message, the instruction execution framework parses the result message to obtain the execution result.

[0193] If there is no result response after the preset time, that is, the instruction execution framework does not receive the result message from the target video application, it means that the electronic device may have closed the target video application. Therefore, the instruction execution framework unbinds the service. The preset time can be, for example, 4s, 5s, etc.

[0194] It can be seen that in the EventManagerService#injectVideoEvent method, the embodiment of the present application will perform service binding, message assembly, message sending, message parsing and processing, service unbinding, etc. Among them, EventManagerService can be understood as the above-mentioned instruction execution framework.

[0195] After the voice command is executed, that is, the video application responds to the user's voice command, the execution result (success or failure) of the broadcast control command can be fed back in real time on the video interface (application interface). Among them, the instruction execution framework parses the result message from the target video application, and after obtaining the execution result, the execution result can be returned to the voice framework. The voice framework determines whether to display the execution result in the application interface. For example, if the execution result received by the voice framework is a failure, the voice framework can interact with the foreground display control and display the prompt information corresponding to the execution failure on the application interface. For another example, if the execution result received by the voice framework is a success, the voice framework may not interact with the foreground display control, that is, the application interface does not display the prompt information corresponding to the successful execution. At this time, the above-mentioned execution result fed back in real time on the video interface is the control state of the video in the video application after the video application responds to the user's voice command (for example, the video is successfully paused).

[0196] In the embodiment of the present application, the application side can implement specific broadcast control capabilities through a customized Service, including operations such as play / pause, fast forward / rewind, rate adjustment, previous / next (short video), and like / favorite / follow. After recognizing the user's intention, the electronic device can send a corresponding message to the customized Service, and the Service implements specific broadcast control behavior based on the message parameters.

[0197] In summary, in the video control method provided in the embodiment of the present application, the electronic device can pre-store the mapping relationship between voice commands and event codes, and after receiving the user's voice information, recognize the corresponding voice command. Then the voice command is mapped to the corresponding event code, and when the target video application is still the focus application, the voice command corresponding to the event code is executed. In this way, the video can be controlled by the user's voice command, and the user operation is more convenient.

[0198] The present application also provides a chip system, such as Fig.12 As shown, the chip system 1200 includes at least one processor 1201 and at least one interface circuit 1202. The processor 1201 and the interface circuit 1202 can be interconnected via lines. For example, the interface circuit 1202 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 1202 can be used to send signals to other devices (such as processor 1201). Exemplarily, the interface circuit 1202 can read instructions stored in the memory and send the instructions to the processor 1201. When the instructions are executed by the processor 1201, the electronic device can perform the various steps in the above embodiments. Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0199] An embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step executed by the mobile phone in the above-mentioned method embodiment.

[0200] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the mobile phone in the above method embodiment.

[0201] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0202] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0203] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0204] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0205] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0206] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A video control method, characterized in that: In a process of playing a video by a target video application in an electronic device, the method includes: The electronic device receives voice information from a user, and if the electronic device selects a target voice instruction corresponding to the voice information from the preset voice instructions corresponding to the target video application, determines a corresponding target instruction code based on the target voice instruction; In a case where the target video application is a current focus application of the electronic device, the electronic device determines that the target video application supports instruction processing; The electronic device controls the video in the target video application based on the target voice instruction corresponding to the target instruction code.

2. The video control method according to claim 1, characterized in that: Before the electronic device receives the user's voice information, the method further includes: When the electronic device detects that the target video application is in a state of playing a video, the interface of the electronic device displays a first prompt message; wherein the first prompt message is used to prompt the user whether to turn on the voice control function; The electronic device receiving the user's voice information includes: The electronic device activates the voice control function in response to the user's touch operation on the first prompt information; The electronic device receives the voice information of the user.

3. The video control method according to claim 1, characterized in that: The voice control function supports the target video application, and before the electronic device receives the user's voice information, it also includes: The electronic device turns on the voice control function in response to the user turning on the control corresponding to the voice control function; The electronic device receiving the user's voice information includes: When it is detected that the target video application is in a state of playing a video, the electronic device receives voice information of a user.

4. The video control method according to any one of claims 1 to 3, characterized in that: The electronic device receives voice information of a user, and if the electronic device selects a target voice instruction corresponding to the voice information from the pre-set voice instructions corresponding to the target video application, then determines a corresponding target instruction code based on the target voice instruction, including: The voice framework of the electronic device receives the voice information of the user; If the voice framework selects a target voice instruction corresponding to the voice information from the preset voice instructions corresponding to the target video application, determine a corresponding target instruction code based on the target voice instruction; Wherein, after determining the corresponding target instruction code based on the target voice instruction, the method further includes: The voice framework sends the target instruction code to the instruction execution framework of the electronic device.

5. The video control method according to claim 4, characterized in that: When the target video application is the current focus application of the electronic device, the electronic device determines that the target video application supports instruction processing, including: The instruction execution framework identifies the current focus application of the electronic device in response to receiving the target instruction code sent by the voice framework, and determines whether the target video application is the current focus application; In a case where the target video application is the current focus application, the instruction execution framework determines that the target video application supports instruction processing.

6. The video control method according to claim 5, characterized in that: The electronic device controls the video in the target video application based on the target voice instruction corresponding to the target instruction code, including: The instruction execution framework constructs a first message, wherein the first message includes the target instruction code; The instruction execution framework sends the first message to the target video application; In response to receiving the first message, the target video application obtains the target instruction code in the target message; The target video application performs corresponding playback control operations on the currently playing video in the target video application based on the target voice instruction corresponding to the target instruction code.

7. The video control method according to claim 6, characterized in that: The instruction execution framework constructs a first message, including: The instruction execution framework determines whether to bind a target service of the target video application; wherein the target service is used to implement a play control operation for a video in the target video application; If the instruction execution framework is not bound to the target service, the instruction execution framework initiates a binding request to the target video application; The target video application returns the proxy object of the target service to the instruction execution framework in response to the received binding request; wherein the proxy object is used to call the target service of the target video application; The instruction execution framework composes the first message in response to receiving the proxy object; or, The instruction execution framework determines whether to bind the target service of the target video application; If the instruction execution framework is bound to the target service, the instruction execution framework constructs the first message.

8. The video control method according to claim 7, characterized in that: The target video application performs corresponding playback control operations on the currently playing video in the target video application based on the target voice command corresponding to the target command code, including: The target video application calls the target service based on the target voice instruction corresponding to the target instruction code, and performs corresponding playback control operations on the currently playing video in the target video application; Wherein, after the target video application performs a corresponding playback control operation on the currently played video in the target video application based on the target instruction code, the method further includes: The target video application obtains the execution result of the target instruction code, and composes a second message based on the execution result; The target video application sends the second message to the instruction execution framework; The instruction execution framework parses the second message in response to receiving the second message to obtain the execution result; The instruction execution framework sends the execution result to the voice framework.

9. The video control method according to claim 8, characterized in that: After the instruction execution framework sends the execution result to the voice framework, the method further includes: In response to the received execution result, the voice framework displays a second prompt message on the interface of the target video application based on the execution result; wherein the second prompt message is used to prompt the user of the execution result corresponding to the target voice instruction.

10. The video control method according to claim 8 or 9, characterized in that: The method further comprises: If the instruction execution framework does not receive the second message within a preset time period, the instruction execution framework unbinds the target service.

11. An electronic device, characterized in that: The electronic device comprises: a communication module, a display screen, a memory and one or more processors; the communication module, the display screen, the memory and the processor are coupled; the memory is used to store computer program code, and the computer program code comprises computer instructions, and when the computer instructions are executed by the electronic device, the electronic device executes the method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed in an electronic device, the electronic device executes the method according to any one of claims 1 to 10.

Images