A method for controlling video quality, an electronic device, and a storage medium.

Abstract

The application provides a video quality control method, an electronic device and a storage medium, and relates to the technical field of terminals. The method comprises the following steps: firstly, obtaining running state information of a terminal device, wherein the running state information of the terminal device comprises temperature information and power information of the terminal device; then, obtaining running scene information of a target video application on the terminal device; wherein the running scene information of the target video application comprises window state information of the target video application; subsequently, whether to perform video quality enhancement on a video of the target video application can be determined by using the running state information of the terminal device and the running scene information of the target video application, and / or the frame rate of the video of the target video application is controlled by using at least one of the running state information of the terminal device and the running scene information of the target video application, so that the video quality can be adaptively controlled according to the change of the window state, and the power consumption and load of the terminal device are optimized.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method for controlling video quality, an electronic device, and a storage medium. Background Technology

[0002] With the continuous breakthroughs in artificial intelligence technology and the increasing prevalence of various terminal devices, more and more users are using various application software functions on these devices to meet their daily needs, such as watching videos or playing games. When users watch videos or make video calls using video software on their terminal devices, the higher the video quality, the higher the power consumption and load required for downloading and buffering on the terminal device.

[0003] Currently, to improve video quality, image enhancement modules or algorithms are typically incorporated into the video decoding section of terminal devices to enhance video quality and improve the user's video viewing experience. However, these image enhancement modules or algorithms often operate in all video scenarios, resulting in wasted power consumption and increased load in some situations. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a video quality control method, electronic device, and storage medium, with the aim of adaptively controlling video quality according to changes in the video window state to optimize terminal device power consumption and load.

[0005] In a first aspect, this application provides a method for controlling video quality. The method includes: firstly acquiring the operating status information of a terminal device, wherein the operating status information of the terminal device includes temperature information and battery information of the terminal device; then acquiring the operating scene information of a target video application on the terminal device, wherein the operating scene information of the target video application includes window status information of the target video application; next, using the operating status information of the terminal device and the operating scene information of the target video application, determining whether to perform quality enhancement control on the video of the target video application, and / or, using at least one of the operating status information of the terminal device and the operating scene information of the target video application, performing frame rate control on the video of the target video application.

[0006] As can be seen, in the above-mentioned video quality control method, when the user makes the running video application a small window or hides it, this embodiment no longer directly performs image quality enhancement processing. Instead, it adaptively controls whether to perform video quality enhancement processing by fusing the temperature and power information of the terminal device and the window state information of the video application. For example, when the video window is a small window, no image quality enhancement processing is performed, and frame dropping processing is performed. Although the video quality is reduced, the power consumption and load of the terminal device can be optimized. When the video window is updated to a non-small window, it means that the user is still paying attention to the video window. At this time, image quality enhancement processing can be resumed without frame dropping processing, thereby improving the video quality and enhancing the user's video viewing experience. This achieves a balance between the power consumption of the terminal device and the video quality.

[0007] In one possible implementation, the system utilizes the operating status information of the terminal device and the operating scenario information of the target video application to determine whether to enhance the video quality of the target video application. This includes: determining whether the target video application needs to perform image quality recognition, obtaining a judgment result; when the judgment result determines that image quality recognition of the video in the target video application is required, the system uses the temperature and power information of the terminal device to determine the status recognition result of the terminal device, and then fuses the status recognition result of the terminal device with the video window status information of the video application used by the user to obtain a recognition result; furthermore, based on the recognition result, the system determines whether to perform image quality enhancement processing on the video of the target video application. This ensures that the system adaptively controls whether to perform video quality enhancement according to changes in the video window status, effectively avoiding wasted power consumption and increased load on the terminal device.

[0008] In one possible implementation, determining whether to enhance the video quality of the target video application based on the recognition result includes: generating an image quality enhancement control command based on the recognition result, and sending the image quality enhancement control command to the video post-processing service unit Vpp so that Vpp can execute the image quality enhancement control command to determine whether to enhance the video quality of the target video application, thereby optimizing the power consumption and load of the terminal device.

[0009] In one possible implementation, the recognition result includes, but is not limited to, globally off, globally on, or on / off of a single application record in the target video application.

[0010] In one possible implementation, frame rate control of the target video application's video is performed using at least one of the terminal device's operating status information and the target video application's operating scene information. This includes: determining whether the target video application's window is small, hidden, or destroyed based on the target video application's window status information, obtaining a determination result, updating the target video application's scene recognition result record based on the determination result, and then performing frame rate control on the target video application's video. This ensures that video frame dropping processing is adaptively performed as the video window status changes, thereby reducing the terminal device's power consumption and load.

[0011] In one possible implementation, based on the judgment result, the scene recognition result record of the target video application is updated and the frame rate of the target video application is controlled. This includes: based on the judgment result, updating the scene recognition result record of the target video application and generating a frame rate control instruction, and sending the frame rate control instruction to the multimedia codec so that the multimedia codec can execute the frame rate control instruction to achieve frame rate control of the target video application, thereby optimizing the power consumption and load of the terminal device.

[0012] In one possible implementation, the target video application on the terminal device refers to the video application that the user is currently making a video call or watching a video, such as common video chat software or video viewing software.

[0013] In one possible implementation, the window state information of the target video application includes, but is not limited to, whether the window of the target video application is a small window or hidden, which is used as the basis for adaptive control of image quality.

[0014] Secondly, this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor is used to call and execute the computer program to implement the video quality control method described in any one of the first aspects above.

[0015] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when run by a processor of an electronic device, is used to implement the video quality control method described in any one of the first aspects above.

[0016] Fourthly, this application provides a computer program product that, when run on a computer, causes the computer to execute the video quality control method as described in any one of the first aspects. Attached Figure Description

[0017] Figure 1 This is one of the scenario illustrations provided in the embodiments of this application;

[0018] Figure 2 A schematic diagram of an electronic device provided in an embodiment of this application;

[0019] Figure 3 A software structure block diagram of an electronic device provided in an embodiment of this application;

[0020] Figure 4 A flowchart illustrating the video quality control method provided in this application embodiment;

[0021] Figure 5 A schematic diagram illustrating the process of determining whether a video application needs to perform image quality recognition, provided in an embodiment of this application.

[0022] Figure 6 A schematic diagram illustrating the process of obtaining temperature and battery information of a terminal device as provided in an embodiment of this application;

[0023] Figure 7 One of the schematic diagrams illustrating the process of obtaining video window state information of a video application used by a user, provided in an embodiment of this application;

[0024] Figure 8 A schematic diagram illustrating the process of fusing and recognizing temperature and battery information of a terminal device and video window status information of a video application used by a user, as provided in this embodiment of the application.

[0025] Figure 9 A schematic diagram illustrating the generation process of image quality enhancement control instructions provided in this application embodiment;

[0026] Figure 10 A schematic diagram illustrating the execution process of the image quality enhancement control command provided in an embodiment of this application;

[0027] Figure 11 A second schematic diagram illustrating the process of obtaining video window status information of a user's video application, provided in an embodiment of this application.

[0028] Figure 12 A schematic diagram illustrating the generation process of frame rate control instructions provided in this application embodiment;

[0029] Figure 13 A schematic diagram illustrating the execution process of the frame rate control instruction provided in this application embodiment;

[0030] Figure 14 This is a schematic diagram illustrating the process of dropping frames according to a frame rate threshold, as provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions "a," "an," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the embodiments of this application, "one or more" refers to one, two, or more; "and / or" describes the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0032] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0033] The "multiple" mentioned in the embodiments of this application refers to two or more. It should be noted that in the description of the embodiments of this application, terms such as "first" and "second" are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance, nor should they be construed as indicating or implying order.

[0034] To enable those skilled in the art to better understand the solution of this application, the application scenario of the technical solution of this application will be described first.

[0035] See Figure 1 The illustration shows a scenario diagram provided by an embodiment of this application.

[0036] In this example scenario, when a user is watching videos or making video calls using certain applications, they might be simultaneously operating other pages while viewing or hiding these video applications in a pop-up window. In this case, the video quality (including image quality and frame rate) of these pop-up or hidden videos is not the user's primary concern. For example... Figure 1As shown in (a), when a user views a video being watched in video app B on their phone in a small window while viewing the interface of app A, the user is not primarily concerned with the video quality of the video playing in app B; they are likely more focused on the content displayed on the interface of app A. Similarly, as... Figure 1 As shown in (b), when a user hides the video being watched in video software B on their phone and looks at the display interface of application software A, the user is not concerned about the picture quality of the video being played in video software B, but is more concerned about the content displayed on the interface of application software A.

[0037] Therefore, in scenarios like the two mentioned above, it's not actually necessary to improve the video quality of the video being played in video software B through the terminal device; that is, the video quality doesn't require enhancement processing. However, currently, in order to provide users with higher-quality videos, terminal devices typically include image enhancement modules or algorithms in their video decoding sections to enhance video quality and improve the user's viewing experience. However, since these image enhancement modules or algorithms are usually effective in all video playback scenarios, this can lead to wasted power consumption and increased load on the terminal device in some scenarios, such as the two mentioned above, and may even cause the device to overheat.

[0038] To overcome the above technical problems, this application provides a video quality control method, electronic device, and storage medium. It can adaptively control video quality when the user minimizes or hides the video window, thereby optimizing terminal device power consumption and load, and improving the user's video viewing experience.

[0039] The video quality control method provided in this application embodiment can be applied to electronic devices such as mobile phones, tablets, televisions, medical instruments, personal digital assistants (PDAs), desktop, laptop, and notebook computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, and wearable devices.

[0040] To enable those skilled in the art to better understand the video quality control method provided in this application, the hardware architecture and software system architecture of the electronic device will be described in detail below.

[0041] See Figure 2 The diagram illustrates an electronic device provided in an embodiment of this application.

[0042] like Figure 2As shown, the electronic device 200 may include a processor 210, a temperature sensor 220, a motor 230, a button 240, an indicator 250, a power management module 260, a battery 261, a charging management module 262, a universal serial bus (USB) interface 263, a camera 270, a display screen 280, an audio module 290, a speaker 290A, a receiver 290B, a microphone 290C, and a headphone jack 290D.

[0043] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 200. In other embodiments of this application, the electronic device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0044] Processor 210 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. The video codec can achieve adaptive control of video quality based on window state changes to optimize power consumption and load on the terminal device.

[0045] The processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store instructions or data that the processor 210 has just used or that are used repeatedly. If the processor 210 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0046] In some embodiments, the processor 210 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0047] Temperature sensor 220 is used to detect the temperature of electronic device 200. In some embodiments, electronic device 100 uses the temperature detected by temperature sensor 220 to execute a temperature processing strategy. For example, when the temperature reported by temperature sensor 220 exceeds a threshold (e.g., 37°C), electronic device 200 reduces the performance of the processor located near temperature sensor 220, such as by not performing video quality enhancement processing, to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is below another threshold, electronic device 200 heats battery 261 to prevent abnormal shutdown of electronic device 200 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, electronic device 200 boosts the output voltage of battery 261 to prevent abnormal shutdown due to low temperature.

[0048] Motor 230 can generate vibration alerts. Motor 230 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations in different applications (such as video playback). The touch vibration feedback effect can also be customized.

[0049] Buttons 240 include a power button, volume buttons, etc. Buttons 240 can be mechanical buttons or touch-sensitive buttons. Electronic device 200 can receive button input and generate key signal inputs related to user settings and function control of electronic device 200. For example, electronic device 200 can receive button input from the user to hide the video software window and generate corresponding key signal inputs.

[0050] USB Interface 263 is an interface compliant with the USB standard specification, specifically including Mini USB, Micro USB, and USB Type-C interfaces. USB Interface 263 can be used to connect a charger to charge electronic device 200, and can also be used for data transfer between electronic device 200 and peripheral devices. It can also be used to connect headphones for playing audio during video calls or video playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0051] The charging management module 262 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 262 receives charging input from the wired charger via a USB interface 263. In some wireless charging embodiments, the charging management module 262 receives wireless charging input via the wireless charging coil of the electronic device 200. While charging the battery 261, the charging management module 262 can also supply power to the electronic device 200 via the power management module 260.

[0052] The power management module 260 connects to the battery 261, the charging management module 262, and the processor 210. The power management module 260 receives input from the battery 261 and / or the charging management module 262, supplying power to the processor 210, display screen 280, camera 270, etc. The power management module 260 can also monitor parameters such as the battery level of the electronic device, using this information as a control basis to determine whether to perform image enhancement processing on the corresponding video application. In some other embodiments, the power management module 260 may also be located within the processor 210. In other embodiments, the power management module 260 and the charging management module 262 may be located in the same device.

[0053] Camera 270 is used to capture still images or videos, such as capturing a user's own video feed during a video call. An object is projected onto a photosensitive element through a lens, generating an optical image. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP (Internet Service Provider) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, electronic device 200 may include one or N cameras 270, where N is a positive integer greater than 1.

[0054] The display screen 280 is used to display images, videos, etc., such as showing a user watching a video or making a video call using an application. The display screen 280 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 200 may include one or N displays 280, where N is a positive integer greater than 1.

[0055] Electronic device 200 implements display functions through a GPU, display screen 280, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 280 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0056] Electronic device 200 can implement audio functions through audio module 290, speaker 290A, receiver 290B, microphone 290C, headphone jack 290D, and application processor. Examples include music playback, video calls, and voice input / output during video playback.

[0057] The audio module 290 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 290 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 290 may be located in the processor 210, or some functional modules of the audio module 290 may be located in the processor 210.

[0058] The speaker 290A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 200 can listen to music or make hands-free calls through the speaker 290A.

[0059] The receiver 290B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 200 receives a telephone call or voice message, the receiver 290B can be brought close to the ear to hear the voice.

[0060] Microphone 290C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 290C, inputting the sound signal into microphone 290C. Electronic device 200 may have at least one microphone 290C. In some embodiments, electronic device 200 may have two microphones 290C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 200 may have three, four, or more microphones 290C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0061] The headphone jack 290D is used to connect wired headphones. The headphone jack 290D can be a USB 263 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0062] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 200. In other embodiments of this application, the electronic device 200 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0063] In addition, the electronic device 200 runs an operating system on top of the aforementioned components. Examples include iOS, Android, and Windows operating systems. Applications can be installed and run on this operating system.

[0064] See Figure 3 It shows a schematic diagram of the software structure of the electronic device provided in the embodiments of this application.

[0065] The software system of electronic device 200 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 200.

[0066] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided from top to bottom into the application layer, framework layer, Android runtime and system libraries, hardware abstraction layer (HAL), and kernel layer.

[0067] The application layer can include a series of application packages. For example... Figure 3 As shown, the application package can include applications such as camera, call, music, video, WLAN, Bluetooth, and scene recognition. The video application uses the Media Player interface to transmit video data streams to the Media Codec in the framework layer and the Codec Abstraction Layer (Codec HAL) in the hardware decoding layer. This allows the Video Post-Processing Service (Vpp Service) to provide a dynamic start / stop interface to control the image enhancement algorithms within the Vpp, and the Media Codec to provide a control interface to control the on / off state of the video frame rate control algorithm (i.e., control the displayed frame rate). The scene recognition application is used to identify the status of the terminal device and perform scene-based identification (such as phone temperature, battery level, application scenario, window status information, etc.) and control of various applications. This allows for scene-based dynamic start / stop (i.e., start / stop of video image enhancement and frame rate control) voting control of videos from applications on the image enhancement whitelist and frame rate control whitelist.

[0068] The framework layer provides the application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 3 As shown, the application framework layer may include a window manager, a content manager, a view system, a media codec, and a dynamic start / stop interface for changing scenarios.

[0069] The window manager is used to manage window applications. It can obtain the screen size, determine if a status bar is present, lock the screen, and capture screenshots, among other things.

[0070] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, the display interface of an application may include views for displaying text and views for displaying video images.

[0071] The Media Codec contains a video frame rate control algorithm to control the video display frame rate. It provides registration information of each application to the scene recognition application through a dynamic start / stop interface that changes the scene, and receives and executes frame rate control instructions sent by the scene recognition application to enable or disable the application video.

[0072] The specific methods for controlling the frame rate of a video using a video frame rate control algorithm may include, but are not limited to, dropping all frames and dropping some frames. The specific implementation method for dropping some frames is not limited and can be processed according to the actual situation and empirical values. For example, it can be processed according to the implementation process of dropping frames according to the frame rate threshold as detailed in the following embodiments.

[0073] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0074] The core library consists of two parts: one part contains the functionalities that the Java language needs to call, and the other part is the Android core library. The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0075] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0076] The Hardware Abstraction Layer (HAL) sits between the Linux kernel and the system runtime layer. It provides a unified encapsulation of lower-level Linux drivers and exposes interfaces to higher layers, hiding the underlying details. It can include, but is not limited to, multimedia hardware codecs (media.hwcodec), codec abstraction layers (Codec HAL), etc. Figure 3 As shown, the codec abstraction layer may include a Video Post-Processing Service (Vpp Service) unit, which utilizes image quality enhancement control algorithms to enhance the image quality of various video applications. Specific methods for enhancing video image quality may include, but are not limited to, resolution enhancement, color enhancement, SDR to HDR dynamic range conversion, super-resolution, and intelligent frame interpolation.

[0077] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0078] The technical solutions involved in the following embodiments can all be implemented in electronic devices with the above-described hardware and software architectures.

[0079] The implementation process of the video quality control method provided in this application will be described in detail below:

[0080] In some embodiments, in order to solve such Figure 1 The present application addresses the issues of power consumption waste and increased load on terminal devices caused by scenarios such as making video applications smaller or hidden, aiming to prevent "overheating" of the terminal device. This application provides a video quality control method that adaptively controls video quality when the user makes the video window smaller or hidden, thereby optimizing terminal device power consumption and load. Figure 4 As shown, the specific implementation process of this video quality control method may include the following steps S401-S403:

[0081] S401: Obtain the operating status information of the terminal device, which includes the temperature information and power information of the terminal device.

[0082] In this embodiment, to optimize the power consumption and load of the terminal device and improve the user's video viewing experience, a whitelist of applications requiring adaptive video quality control can be pre-set. This whitelist consists of at least one preset video application (such as video calling and video playback applications). That is, when a user uses any video application in the whitelist (hereinafter referred to as the target video application) to make a video call or watch a video, steps S401-S402 can be executed to adaptively control the video quality according to the changes in the video window state of the target video application (such as full-screen video, small window video, or hidden video). This includes, but is not limited to, whether to enhance the image quality and control the transmission frame rate, in order to optimize the power consumption and load of the terminal device.

[0083] Among them, the scenarios for video pop-ups include, but are not limited to: video chat pop-ups, floating pop-ups built into video playback applications (Pip picture-in-picture pop-ups are determined by an area threshold), and mini-windows in self-developed floating pop-ups.

[0084] The scenarios for video hiding include, but are not limited to: hiding the small window of a video chat, making the video chat invisible when it is closed and exited to the background, and other video hiding scenarios that can be expanded in the future.

[0085] Furthermore, when it is detected that a user is using any video application in the whitelist to make a video call or watch a video, in order to achieve adaptive control of the video quality, the operating status information of the terminal device can first be obtained to execute subsequent step S403. The operating status information of the terminal device may include, but is not limited to, the temperature information and power information of the terminal device. For example, the temperature information of the device can be obtained using a pre-installed temperature sensor on the terminal device, and the power information of the device can be obtained using a pre-installed power management module on the terminal device. This is used to determine which type of video quality control the current operating status of the terminal device can support for the preset video application, i.e., at least one of the following video quality control methods: only image quality enhancement, only frame dropping processing, both image quality enhancement and frame dropping processing, or neither image quality enhancement nor frame dropping processing. This reduces the power consumption and load of the terminal device, thus avoiding overheating.

[0086] S402: Obtain the running scene information of the target video application on the terminal device; wherein, the running scene information of the target video application includes the window state information of the target video application.

[0087] In this embodiment, in order to optimize the power consumption and load of the terminal device, it is also necessary to obtain the running scenario information of the target video application on the terminal device for subsequent step S403. The running scenario information of the target video application may include the window state information of the target video application, which may include, but is not limited to, the small window and the hidden window status of the video window.

[0088] The criteria for determining a video window to be considered a small window are not limited and can be set based on actual conditions and experience. For example, a video window can be considered a small window if its area occupies less than a preset percentage threshold of the entire display screen on the terminal device. The preset percentage threshold is also not limited and can be set based on actual conditions and experience. For example, it can be set to 1 / 4, in which case the video window will be considered a small window if its area occupies less than 1 / 4 of the entire display screen on the terminal device.

[0089] S403: Using the operating status information of the terminal device and the operating scenario information of the target video application, determine whether to enhance the video quality of the target video application, and / or, using at least one of the operating status information of the terminal device and the operating scenario information of the target video application, perform frame rate control on the video of the target video application to achieve a balance between power consumption of the terminal device and video quality.

[0090] In this embodiment, after obtaining the operating status information of the terminal device, such as temperature and power consumption, through step S401, and the operating scenario information, such as the window status of the preset video application, through step S402, in order to optimize the power consumption and load of the terminal device, the operating status information of the terminal device and the operating scenario information of the preset video application can be used to generate and issue control instructions (such as global off, global on, single video application record on / off, no control requirement, etc.) on or off for the video of the preset video application. The corresponding video quality control processing is executed according to the control instructions to achieve a balance between the power consumption and video quality of the terminal device.

[0091] In this way, when a user is viewing a video or making a video call while simultaneously operating other pages, the video quality of these windowed or hidden videos can be adaptively controlled by executing the steps S401-S403 above. This can optimize the power consumption and load of the terminal device and improve the user's video viewing experience.

[0092] Next, this application embodiment will describe in detail the process of determining whether the video should be enhanced in terms of image quality. In some embodiments, the specific implementation process of the method for determining whether the video should be enhanced in terms of image quality may include the following steps S501-S506:

[0093] S501: Determine whether a video application that the user is watching or making a video call needs to perform image quality recognition, and obtain the judgment result.

[0094] In this embodiment, in order to achieve adaptive control over whether to enhance video quality, it is first possible to determine whether the video application the user is watching or making a video call needs to perform quality recognition, and obtain a determination result. The determination process may include... Figure 5 Steps S5011-S5015 are as follows: First, a determination can be initiated regarding whether image quality recognition is required. This involves detecting whether the user is watching a video or making a video call through a video application. If so, it is further determined whether the video application belongs to the image quality enhancement recognition whitelist. If not, the determination result is that image quality recognition is not required. Conversely, if it belongs to the image quality enhancement recognition whitelist, it is necessary to determine whether the video application is a designated video application (referring to video applications that are pre-defined according to actual conditions and must be subject to image quality enhancement control; the specific content is not limited) or whether image quality enhancement in the settings menu is enabled. If the video application is not a designated video application and image quality enhancement in the settings menu is disabled, the determination result is also that image quality recognition is not required. However, if the video application is a designated video application or image quality enhancement in the settings menu is enabled, the determination result is that image quality recognition is required.

[0095] The specific contents of the image quality enhancement recognition whitelist are not limited and can be preset based on actual conditions and experience. For designated video applications (such as a video chat software) that are not controlled by the settings menu switch status, if the designated video application is running in the foreground of the terminal device, the judgment result requiring image quality recognition can be returned directly.

[0096] S502: When it is determined from the judgment result that the video in the video application needs to be image quality recognized, the temperature information and power information of the terminal device are obtained, and the temperature information and power information of the terminal device are used to determine the status recognition result of the terminal device.

[0097] In this embodiment, when it is determined from the judgment result that the video in the video application needs to be image quality identified, it is further necessary to obtain the temperature information and power information of the terminal device, and perform fusion processing on the two to determine the current state identification result of the terminal device, that is, to determine whether to perform global enhancement of video image quality.

[0098] The determination process may include Figure 6 Steps A1, A2 (A21-A26), A3 (A31-A34), and A4 (A41-A44) are as follows: When it is determined that the temperature of the acquired terminal device is greater than the preset high temperature threshold (high_temp, the specific value is not limited and can be set according to the actual situation and experience, such as setting high_temp to 37℃), the current temperature control state is set to off, and the temperature control voting enhancement is globally turned off; when it is determined that the temperature of the acquired terminal device is lower than the preset low temperature threshold (low_temp, the specific value is not limited and can be set according to the actual situation and experience, such as setting low_temp to 35℃), the current temperature control state is set to on, and the temperature control voting enhancement is globally turned on; when it is determined that the temperature of the acquired terminal device is between the preset low temperature threshold (low_temp, such as 35℃) and the preset high temperature threshold (high_temp, such as 37℃), the previous temperature control voting state is retained.

[0099] When it is determined that the battery level of the terminal device is lower than the preset low battery threshold (battery_low, the specific value is not limited and can be set according to the actual situation and experience, such as setting battery_low to 10%) and is not connected to a charger, the battery voting result is set to globally disable enhancement; when it is determined that the battery level of the terminal device is not lower than the preset low battery threshold (battery_low), or is lower than the preset low battery threshold but is connected to a charger, the battery voting result is set to globally enable enhancement.

[0100] Based on this, when both temperature control and power consumption votes result in "enhancement on", the combined result is "enhancement on globally". When either temperature control or power consumption votes result in "enhancement off", the combined result is "enhancement off globally". In this way, whether the terminal device is low on power or at a high temperature, the power consumption can be reduced or the temperature can continue to rise by disabling the video quality enhancement function.

[0101] S503: Obtain the video window status information of the video application used by the user on the terminal device.

[0102] In this embodiment, to achieve adaptive control over whether to enhance video quality, it is also necessary to obtain the video window state information of the video application used by the user on the terminal device, and determine whether the video window is small or hidden. If not, it is further necessary to determine whether the video window has been destroyed. The specific determination process may include... Figure 7 Steps S71-S77 are shown.

[0103] S504: The terminal device's status recognition result and the video window status information of the video application used by the user are fused and recognized to obtain the recognition result.

[0104] In this embodiment, after obtaining the temperature and battery information of the terminal device in step S502 and determining the status identification result of the terminal device, and after obtaining the video window status information of the video application used by the user in step S503 and determining whether the video window is a small window or hidden, this information can be further fused and identified to obtain the identification result, including globally off, globally on, and single application record on / off, etc. The specific identification process may include Figure 8 Steps S81-S88 are shown.

[0105] First, it can be determined whether the scene recognition status of the video application has been updated. If so, the scene recognition result record needs to be updated. The result record for this application is <application package name, this application's image quality switch>, and this image quality switch is only affected by changes in the application scene.

[0106] S505: Generate image quality enhancement control instructions based on the recognition results.

[0107] In this embodiment, after obtaining the recognition result (global off, global on, single application record on / off, etc.) through step S504, a picture quality enhancement control command can be generated and sent to the Vpp module. The specific implementation process may include... Figure 9 Steps S91-S94 are shown.

[0108] The parameter format for the image quality enhancement control command can be: Image Quality Enhancement: Globally Enabled; Enhancement: Number of Items: Package Name On / Off Status; Package Name On / Off Status; ...

[0109] For example, the parameter format for the recognition result being globally off can be: Image quality enhancement: Globally off enhancement.

[0110] The recognition result is a single application record. The parameter format can be: Image quality enhancement: Globally enabled. Enhancement: 1: Package name on status.

[0111] The recognition result can be in the following format: Image quality enhancement: Globally enabled enhancement: Number of entries: Package name on / off status; Package name on / off status; ...

[0112] It should be noted that the optional interface design provided in this application can use a unified interface or a single independent interface. Considering that the control is not high-frequency, a separate control interface design that sends image enhancement control commands to the VPP is also possible, as shown in the following example:

[0113] Globally enable `a(true, null)`, globally disable `a(false, null)`; individually enable `a(true,"packageName")`, individually disable `a(false,"packageName")`. When there are multiple application entries, the individual application enable / disable interface can be called multiple times.

[0114] S506: The video post-processing service unit (Vpp) executes the image quality enhancement control instruction to determine whether to enhance the image quality of the video.

[0115] In this embodiment, after generating and sending the image quality enhancement control command to the Vpp module in step S505, the Vpp module can further execute the image quality enhancement control command, determine whether to enhance the video image quality, and when it is determined that the video needs image quality enhancement, use a pre-stored video image quality enhancement algorithm to perform image quality enhancement processing on the video, such as resolution enhancement, color enhancement, SDR to HDR dynamic range conversion, super-resolution, intelligent frame interpolation, and other image quality enhancement processing operations. The specific implementation process may include... Figure 10 The steps B1-B5 are shown. It should be noted that the Vpp module needs to pre-cache the state of the global switch. When the global switch is off, even if a new stream is created using the whitelist application, the post-processing algorithm must be disabled. When the global switch is on, the post-processing algorithm for newly created streams using the whitelist application is enabled by default.

[0116] Next, this application embodiment will describe in detail the implementation process of frame rate control for a preset video application. In some embodiments, the specific implementation process of the method for frame rate control for a preset video application may include the following steps S601-S603:

[0117] S601: Obtain the video window status information of the video application used by the user on the terminal device, perform recognition processing, and obtain the recognition result.

[0118] In this embodiment, in order to achieve adaptive processing of video frame rate control, it is first necessary to identify the frame rate control scenario to determine whether the video application used by the user has a frame rate control recognition requirement. If so, after the scenario recognition is performed, the recognition result can be returned, the application scenario recognition result record can be updated, and a frame rate control command can be generated and issued.

[0119] The process may include Figure 11 Steps C1-C9, as shown, specifically involve: after obtaining the video window status information of the video application used by the user on the terminal device, further determining whether the video window is a small window, hidden, or destroyed, and obtaining the corresponding recognition result (i.e., the video window is a small window, hidden, not a small window, or the window is destroyed). Scenarios for small video windows include, but are not limited to: small video chat windows, floating small windows built into video playback applications (Pip picture-in-picture small windows are determined by an area threshold), and mini windows in self-developed floating windows. Scenarios for hidden videos include, but are not limited to: hiding small video chat windows, making video chat invisible when closing the floating window and exiting to the background, and other video hiding scenarios that can be expanded in the future. Non-small window scenarios include all scenarios other than small video windows and hidden videos.

[0120] S602: Based on the recognition results, update the application scenario recognition result record and generate frame rate control instructions.

[0121] In this embodiment, after obtaining the recognition result (i.e., the changes in the video window of the video application) through step S601, the application's records (package name and frame rate control on / off state) can be further deleted, added, or updated based on the recognition result. A frame rate control command is then generated, and this control command is sent to the MediaCodec module. The specific implementation process may include... Figure 12 The steps D1-D11 are shown.

[0122] The frame rate threshold (denoted by M) represents the minimum frame rate value that the terminal device provides to the user as a quality guarantee. It is determined by performance testing by professional visual engineers. It can be pre-configured in the frame rate control whitelist configuration file, and the frame rate threshold M can be different for different video applications. The specific value of the frame rate threshold M is not limited; it can be 15 frames per second (fps), 13 fps, etc.

[0123] The parameter format for the frame rate control command can be: frame rate control: packet name: switch status: whether to drop all frames: frame rate threshold M.

[0124] For example, the format for disabling the transmission frame rate parameter can be: frame rate control: packet name: off status.

[0125] The format for enabling frame rate control can be: frame rate control: package name: on status: not all dropped: frame rate threshold M.

[0126] To enable frame rate control and drop all frames, the format can be: frame rate control:packet name:on status:drop all frames.

[0127] S603: The Media Codec executes frame rate control instructions to control the frame rate of the video.

[0128] In this embodiment, after generating and sending a frame rate control command to the MediaCodec module in step S602, the MediaCodec module can further parse the command immediately upon receiving it and execute the corresponding processing logic to achieve frame rate control of the video. The specific implementation process may include... Figure 13 The steps E1-E7 are shown. The frame rate control instructions include three processing logics: First, disable frame dropping logic, meaning no frames are dropped, and the video window is not a small window. Second, drop all frames, and the video window is hidden. Third, drop frames according to the frame rate threshold, and the video window is a small window.

[0129] The detailed process of dropping frames according to a frame rate threshold may include... Figure 14Steps F1-F11 are as follows: First, the current frame rate (fps) and frame dropping rate (N) are calculated and updated based on the frame index (frameIndex), the time difference between two frame recordings (T2-T1), and the frame rate threshold parameter M. It is then determined whether to drop frames evenly according to the frame dropping rate. If the current frame rate is lower than the threshold, no frame dropping is performed. Next, the arrival time (ms) of the current frame is recorded every N frames transmitted (e.g., the initial value N=10 after receiving a command), and the current frame rate is calculated based on the time difference with the previous record. Then, when the frame index frameIndex = 0 and the frame rate fps is greater than the threshold M, the current frame is discarded, and the frame dropping rate N = (int)((float)fps / (fps-M)+0.5) is dynamically updated to ensure that the frame rate after frame dropping is close to the threshold M and not lower than M.

[0130] Furthermore, this application also provides an electronic device. For details regarding the hardware structure and software framework of the electronic device, please refer to... Figure 2 and Figure 3 The corresponding explanation is as follows: The electronic device includes a memory and a processor. The memory stores a computer program, and the processor calls and executes the computer program to implement the video quality control method provided in the above description.

[0131] This application also provides a computer-readable storage medium storing a computer program thereon, which, when run by the processor of a terminal device, is used to implement the video quality control method provided above.

[0132] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A video quality control method characterized by comprising: The method comprises: obtaining running state information of a terminal device, wherein the running state information of the terminal device comprises temperature information and power information of the terminal device; obtaining running scene information of a target video application on the terminal device, wherein the running scene information of the target video application comprises window state information of the target video application, and the window state information of the target video application comprises that a window of the target video application is a small window or hidden; determining whether the target video application needs to perform quality recognition, to obtain a determination result; when it is determined according to the determination result that the video in the target video application needs to perform quality recognition, determining a state recognition result of the terminal device by using the temperature information and the power information of the terminal device; fusing the state recognition result of the terminal device and video window state information of a video application program used by a user to obtain a recognition result; determining whether to perform quality enhancement control on the video of the target video application according to the recognition result, and performing frame rate control on the video of the target video application by using the running state information of the terminal device and the running scene information of the target video application.

2. The method of claim 1, wherein, The determining whether to perform quality enhancement control on the video of the target video application according to the recognition result comprises: generating a quality enhancement control instruction according to the recognition result, and delivering the quality enhancement control instruction to a video post-processing service unit Vpp, so that the Vpp executes the quality enhancement control instruction to determine whether to perform quality enhancement on the video of the target video application.

3. The method according to any of claims 1-2, characterized in that, The recognition result comprises global off, global on, or single application record on / off in the target video application.

4. The method of claim 1, wherein, The performing frame rate control on the video of the target video application by using the running state information of the terminal device and the running scene information of the target video application comprises: determining whether the window of the target video application is a small window or hidden or destroyed according to the window state information of the target video application, to obtain a determination result; updating a scene recognition result record of the target video application and performing frame rate control on the video of the target video application according to the determination result.

5. The method of claim 4, wherein, The updating the scene recognition result record of the target video application and performing frame rate control on the video of the target video application according to the determination result comprises: updating the scene recognition result record of the target video application and generating a frame rate control instruction according to the determination result, and delivering the frame rate control instruction to a multimedia codec, so that the multimedia codec executes the frame rate control instruction to perform frame rate control on the video of the target video application.

6. The method of claim 1, wherein, The target video application on the terminal device is a video application in which a current user is performing a video call or watching a video.

7. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor is configured to invoke and execute the computer program to implement the method in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the electronic device to implement the method in any one of claims 1-6.

Images