Video processing method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides a video processing method and device, electronic equipment, a computer readable storage medium and a computer program product. The method is applied to a simulator in a host environment, and the simulator is used for simulating and running a video application program in a virtual environment in the host environment. The method comprises the following steps: receiving decoding configuration parameters sent by a proxy decoder running in a virtual environment; when it is detected that a target hardware decoder adapted to the decoding configuration parameters exists in the host environment, determining a decoding instance of the target hardware decoder in the host environment; when first video code stream data which is sent by the proxy decoder and comes from a video application program is received, decoding the first video code stream data through a decoding instance of the target hardware decoder to obtain first decoded data; and sending the first decoded data to a proxy decoder, so as to send the first decoded data to a video application program for video playing. According to the invention, the video playing fluency of the video application program can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a video processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid development of mobile internet and virtualization technology, running mobile operating systems on personal computers or cloud servers through virtualization platforms such as emulators has become a common application scenario. Users increasingly rely on such platforms to run various multimedia applications, especially video applications such as long video-on-demand, live streaming, and short video applications, which places high demands on the video processing capabilities of virtualization platforms. Therefore, how to improve video processing efficiency in a complex virtualization operating system is an urgent problem to be solved in this field. Summary of the Invention

[0003] This application provides a video processing method, apparatus, electronic device, computer-readable storage medium, and computer program product that can directly decode video through a hardware decoder instance in the host environment when a video application in a virtual environment plays video, reducing the resource consumption of the video application's central processing unit and thus improving video decoding efficiency to enhance the smoothness of video playback in the video application.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a video processing method applied to an emulator in a host environment. The emulator is used to simulate and run a video application in a virtual environment within the host environment. The method includes: receiving decoding configuration parameters for the video application sent by a proxy decoder running in the virtual environment; wherein the proxy decoder is invoked by the video application in response to a video playback operation; when a target hardware decoder adapted to the decoding configuration parameters is detected in the host environment, determining a decoding instance of the target hardware decoder in the host environment; when receiving first video stream data originating from the video application sent by the proxy decoder, decoding the first video stream data using the decoding instance of the target hardware decoder to obtain first decoded data; and sending the first decoded data to the proxy decoder to send the first decoded data to the video application for video playback via the proxy decoder.

[0005] This application provides a video processing apparatus, the apparatus comprising: The parameter receiving module is used to receive decoding configuration parameters for the video application sent by the proxy decoder running in the virtual environment; wherein the video application in the virtual environment is simulated by an emulator running in the host environment, and the proxy decoder is invoked by the video application in response to video playback operations; The instance creation module is used to determine the decoding instance of the target hardware decoder in the host environment when a target hardware decoder that adapts to the decoding configuration parameters is detected in the host environment. The decoding module is used to decode the first video stream data originating from the video application by the target hardware decoder when it receives the first video stream data sent by the proxy decoder, in order to obtain the first decoded data. The sending module is used to send the first decoded data to the proxy decoder, so that the first decoded data can be sent to the video application for video playback through the proxy decoder.

[0006] In the above scheme, the parameter receiving module is further configured to, in response to receiving an initialization request sent by the proxy decoder, parse the initialization request to obtain the decoding configuration parameters; wherein, the proxy decoder is created based on the startup operation of the video application, and the initialization request is generated by the video application when it first responds to the video playback operation and calls the proxy decoder.

[0007] In the above scheme, the instance creation module is further configured to obtain multiple hardware decoders available in the host environment and create a decoding instance for each hardware decoder; determine the decoding capability information of the decoding instance of each hardware decoder, and detect whether each hardware decoder is compatible with the decoding configuration parameters based on the decoding capability information, and obtain the detection result; based on the detection result, select the target hardware decoder that is compatible with the decoding configuration parameters from the multiple hardware decoders, and retain the decoding instance of the target hardware decoder.

[0008] In the above scheme, the instance creation module is further configured to, when there are multiple hardware decoders that adapt to the decoding configuration parameters among the multiple hardware decoders, obtain the load status of each hardware decoder that adapts to the decoding configuration parameters; and filter out the target hardware decoder whose load status is lower than the load threshold among the multiple hardware decoders that adapt to the decoding configuration parameters.

[0009] In the above scheme, before the first video stream data is decoded by the decoding instance of the target hardware decoder, the instance creation module is further configured to send a first prompt message to the proxy decoder, so that the proxy decoder prompts the video application to send the first video stream data to the proxy decoder based on the first prompt message, and sends the first video stream data to the simulator through the proxy decoder; wherein, the first prompt message indicates that the decoding instance of the target hardware decoder has been successfully created.

[0010] In the above scheme, before decoding the first video stream data through the decoding instance of the target hardware decoder, the decoding module is further configured to perform format conversion on the first video stream data to obtain the first video stream data in a target data format; the target data format is a data format supported by the decoding instance of the target hardware decoder; and the first video stream data in the target data format is decoded through the decoding instance of the target hardware decoder.

[0011] In the above scheme, the sending module is further configured to receive a video acquisition request originating from the video application sent by the proxy decoder; parse the video acquisition request to obtain the target video identifier and target color format of the target video requested by the video application; when the video identifier corresponding to the first decoded data is the target video identifier, determine the color format of the first decoded data; when the color format of the first decoded data is different from the target color format, adjust the color format of the first decoded data to the target color format, and send the first decoded data of the target color format to the proxy decoder.

[0012] In the above scheme, the sending module is further configured to create a shader component in the host environment; bind the first decoded data as the input texture of the shader component, and call the decoding instance of the target hardware decoder to run the shader component, so as to perform color space conversion on the input texture through the shader component to generate first decoded data that conforms to the target color format.

[0013] In the above scheme, the sending module is further configured to send a second prompt message to the proxy decoder when the video identifier corresponding to the first decoded data is different from the target video identifier, so as to prompt the proxy decoder to send the second video stream data of the target video to the simulator through the second prompt message; wherein, the second prompt message indicates that there is no decoded data of the target video, and the second video stream data corresponds to the target video identifier; the decoding module is further configured to, when receiving the second video stream data, decode the second video stream data through the decoding instance of the target hardware decoder to obtain the second decoded data, and send the second decoded data to the proxy decoder through the sending module, so as to send the second decoded data to the video application for video playback through the proxy decoder.

[0014] In the above scheme, the first decoded data is stored in a decoded data queue, and the host environment is configured with input processing threads and output processing threads running in parallel; the decoding module is also used to call the input processing thread to send the first video stream data to the decoding instance of the target hardware decoder, and to decode the first video stream data through the decoding instance of the target hardware decoder; and to call the output processing thread to store the first decoded data in the decoded data queue.

[0015] In the above scheme, after receiving the decoding configuration parameters for the video application sent by the proxy decoder running in the virtual environment, the instance creation module is further configured to create a decoding instance of the software decoder in the host environment when it is detected that there is no hardware decoder adapted to the decoding configuration parameters in the host environment, or when the total load of all hardware decoders in the host environment exceeds the load threshold; the decoding module is further configured to decode the first video stream data originating from the video application sent by the proxy decoder, and obtain third decoded data by the decoding instance of the software decoder; the sending module is further configured to send the third decoded data to the proxy decoder, so that the proxy decoder can send the third decoded data to the video application for video playback.

[0016] In the above scheme, the decoding module is further configured to detect the running status of the decoding instance of the target hardware decoder in real time during the decoding of the first video stream data by the decoding instance of the target hardware decoder; when the running status indicates that the decoding instance of the target hardware decoder has encountered a decoding abnormality, the decoding instance of the target hardware decoder is deregistered; the instance creation module is further configured to create a decoding instance of the software decoder in the host environment, so as to decode the first video stream data by the decoding instance of the software decoder.

[0017] In the above scheme, the sending module is further configured to call a pre-trained feature extraction model to extract features from the first decoded data to obtain decoded features; perform enhancement processing on the first decoded data based on the decoded features to obtain enhanced decoded data; and send the enhanced decoded data to the proxy decoder so that the enhanced decoded data can be sent to the video application for video playback through the proxy decoder.

[0018] In the above scheme, the video processing device further includes a deregistration module, which is used to deregister the decoding instance of the target hardware decoder when receiving a shutdown prompt message sent by the proxy decoder for the video application; when the deregistration of the decoding instance of the target hardware decoder is completed, a third prompt message is sent to the proxy decoder to deregister the proxy decoder through the third prompt message; wherein, the third prompt message indicates that the deregistration of the decoding instance of the target hardware decoder is successful.

[0019] In the above scheme, the host environment is configured with shared memory shared with the virtual environment; the sending module is further configured to store the first decoded data in the shared memory and generate a corresponding data index identifier; send the data index identifier to the proxy decoder so that the proxy decoder reads the first decoded data from the shared memory according to the data index identifier.

[0020] In the above scheme, the first decoded data is in a graphics texture format; the sending module is further configured to store the first decoded data in the graphics buffer of the target hardware decoder; map the graphics buffer to the shared memory; or, transmit the first decoded data from the graphics buffer to the shared memory through the decoding instance of the target hardware decoder.

[0021] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the video processing method provided in the embodiments of this application.

[0022] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the video processing method provided in this application when executed by a processor.

[0023] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the video processing method provided in this application.

[0024] The embodiments of this application have the following beneficial effects: In this embodiment, when the emulator simulates a video application in a virtual environment within a host environment, and receives decoding configuration parameters for the video application from a proxy decoder (called by the video application in response to video playback operations), if a target hardware decoder matching the decoding configuration parameters is detected in the host environment, the emulator determines the decoding instance of the target hardware decoder in the host environment. Upon receiving the first video stream data originating from the video application from the proxy decoder, the emulator decodes the first video stream data using the decoding instance of the target hardware decoder to obtain the first decoded data. Finally, the emulator returns the first decoded data to the proxy decoder so that the video application can play video based on the first decoded data. This method enables video decoding to be performed directly through the decoding instance of the hardware decoder in the host environment when the video application in the virtual environment plays video, reducing the resource consumption of the video application's central processing unit and thus improving video decoding efficiency, thereby enhancing the smoothness of video playback in the video application. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the architecture of the video processing system provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Figure 3 This is a first flowchart illustrating the video processing method provided in this application embodiment; Figure 4 This is a flowchart illustrating the method for obtaining decoding configuration parameters provided in an embodiment of this application; Figure 5 This is a flowchart illustrating the method for determining a decoding instance provided in an embodiment of this application; Figure 6 This is a flowchart illustrating the method for sending the first decoded data provided in an embodiment of this application; Figure 7 This is a second flowchart illustrating the video processing method provided in the embodiments of this application; Figure 8 This is a schematic diagram of a mobile phone emulator simulating the running of a video application provided in an embodiment of this application; Figure 9 This is a schematic diagram of the task manager provided in an embodiment of this application; Figure 10 This is a schematic diagram of multi-terminal interaction provided in the embodiments of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0029] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0030] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0031] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0032] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0033] 1) Responding to: used to indicate the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0034] 2) Client, also known as user terminal, refers to the program that provides local services to users in contrast to the server. Except for some applications that can only run locally, it is generally installed on ordinary client machines and needs to work in conjunction with the server. That is, there needs to be a corresponding server and service program in the network to provide the corresponding services. Thus, a specific communication connection needs to be established between the client and the server to ensure the normal operation of the application, such as a video client.

[0035] 3) Host environment: Deployed at the main control operating system level on the terminal or server, it directly controls and manages the underlying core computing resources, including the central processing unit (CPU), graphics processing unit (GPU), and artificial intelligence acceleration chip. It is responsible for establishing efficient cross-domain communication channels to receive video bitstream data to be processed, dynamically scheduling underlying resources to create or destroy decoding instances according to control instructions, running pre-trained feature extraction models and enhancement processing algorithms to improve the image quality of the decoded image data, and performing real-time detection and anomaly management of the overall decoding process.

[0036] 4) Virtual environment: A logically isolated software runtime space built on the underlying physical platform. It is designed to simulate specific operating system behaviors to be compatible with various video applications. Internally, it can integrate a proxy decoder to intercept video playback requests and raw data sent by applications. It transmits the data to the external execution layer for processing through preset shared memory or communication protocols. After receiving the processed ordinary or enhanced image data, it completes the final image rendering and audio synchronization, thereby providing users with a simulation platform with a seamless interactive experience.

[0037] 5) Hardware decoder: A dedicated integrated circuit or functional module for video signal processing, integrated into the core circuit of a physical device (terminal or server). It has efficient parallel computing capabilities and can create independent decoding instances in response to external commands. It can accelerate the decompression of high-resolution, high-frame-rate or specific encoding format video stream data with low power consumption and low latency. At the same time, it can be forcibly deregistered by the system and its occupied video memory and handle resources can be reclaimed when abnormalities such as deadlock or timeout occur during operation or when the business session ends. It is a key physical component for achieving high-performance video playback.

[0038] 6) A software decoder, a video decompression algorithm library written and running based on the instruction set of a general-purpose computing unit, has broad format compatibility and portability, and does not depend on specific dedicated acceleration circuits. In this application, it is mainly used as a logic processing unit that is dynamically loaded and instantiated in emergency scenarios such as failure of the preferred high-performance decoding scheme, resource exhaustion, or response timeout, and takes over the processing tasks of video bitstream data through pure algorithmic logic to ensure that the continuity and stability of video playback services can still be maintained when no dedicated hardware resources are available.

[0039] This application provides a video processing method, apparatus, electronic device, computer-readable storage medium, and computer program product that can directly decode video through a hardware decoder instance in the host environment when a video application in a virtual environment plays video, reducing the resource consumption of the video application's central processing unit and thus improving video decoding efficiency to enhance the smoothness of video playback in the video application.

[0040] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the video processing system 100 provided in the embodiments of this application. In order to support a video processing application, the terminal (exemplarily shown as the first terminal 400-1 and the second terminal 400-2) connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0041] The terminal can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals. Applications or clients can be installed on the terminal, such as multimedia applications (or multimedia clients), learning applications (or learning clients), game applications (or game clients), navigation applications (or navigation clients), live streaming applications (or live streaming clients), etc. These applications or clients can all support video playback; in this application, applications or clients supporting video playback are collectively referred to as video applications. Server 200 is the backend server corresponding to the application or client. It can be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It can also be a cloud application server providing basic cloud computing services such as cloud application services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal and server can be directly or indirectly connected via wired or wireless communication; this embodiment does not impose any restrictions.

[0042] In practical applications, the first terminal 400-1 and the second terminal 400-2 differ in type and operating environment (the first terminal 400-1 operates in the first environment, and the second terminal 400-2 operates in the second environment). The first terminal 400-1 can be configured with an emulator to simulate the video application of the second environment within the first environment. Here, the first environment is the host environment, and the second environment is the virtual environment. For example, the first terminal 400-1 can be a laptop running a Windows operating system, and the second terminal 400-2 can be a smartphone running an Android operating system. The emulator configured in the first terminal 400-1 runs in the host environment (i.e., the first environment, such as the Windows environment). The emulator can be used in the host environment of the first terminal 400-1 to simulate the video application in the virtual environment of the second terminal 400-2. The emulator can take the form of a product; for example, it can be an application distribution client. The distribution client's interface displays a startup entry for running the virtual environment, or it can directly display a startup entry adapted to the virtual environment.

[0043] In one scenario, a user can launch an emulator (such as a distribution client) from a first terminal 400-1. Responding to the launch operation of the emulator (such as the distribution client), the first terminal 400-1 runs the emulator (such as the distribution client) in a host environment (i.e., the first runtime environment, such as a Windows runtime environment) and displays the emulator's first interface. This first interface displays a launch entry point for running a virtual environment. If the user runs the virtual environment through the launch entry point in the first interface, the emulator responds to the launch operation triggered by the launch entry point and simulates the running of a mobile virtual machine in the host environment. The mobile virtual machine is the second runtime environment, or virtual environment. The interface of the mobile virtual machine can display various video applications running in the virtual environment. The user can launch any video application from the mobile virtual machine in the first terminal 400-1. Correspondingly, in response to the launch operation of a video application, the mobile virtual machine can create a proxy decoder for that video application.

[0044] In another scenario, the user can launch an emulator (such as a distribution client) from the first terminal 400-1. In response to the launch operation of the emulator (such as a distribution client), the first terminal 400-1 runs the emulator (such as a distribution client) in the host environment (i.e., the first runtime environment, such as a Windows runtime environment) and displays the first interface of the emulator. The first interface displays a launch entry for a video application adapted to the virtual environment. If the user runs the video application through the launch entry in the first interface, the emulator can respond to the launch operation of the video application triggered by the launch entry, directly simulate the running of the video application in the virtual environment in the host environment, and create a proxy decoder for the video application.

[0045] Regardless of the method used to simulate the video application in the virtual environment within the host environment, when a user first triggers a video playback operation in the video application, the video application, in its initial response, can invoke the corresponding proxy decoder and generate an initialization request carrying decoding configuration parameters. Simultaneously, it sends the initialization request to the proxy decoder, which then forwards it to the emulator. Subsequently, the emulator, upon receiving the initialization request from the proxy decoder, parses it to obtain the decoding configuration parameters. When the emulator detects a target hardware decoder in the host environment that matches the decoding configuration parameters, it determines the decoding instance of the target hardware decoder within the host environment. When it receives the first video stream data originating from the video application from the proxy decoder, it decodes the first video stream data using the decoding instance of the target hardware decoder to obtain the first decoded data. This first decoded data is then sent to the proxy decoder, which then forwards it to the video application for video playback.

[0046] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 2 The illustrated electronic device 400 includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the electronic device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0047] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0048] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0049] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0050] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0051] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0052] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc. Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with user interface 430 (e.g., a display screen, a speaker, etc.). The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0053] In some embodiments, the video processing apparatus provided in this application can be implemented in software. Figure 2 A video processing apparatus 455 stored in memory 450 is shown. This apparatus can be software in the form of programs and plugins, and includes the following software modules: a parameter receiving module 4551, an instance creation module 4552, a decoding module 4553, and a transmitting module 4554. These modules are logically connected and can therefore be arbitrarily combined or further separated according to the functions they implement. The functions of each module will be described below.

[0054] In other embodiments, the video processing apparatus provided in this application can be implemented in hardware. As an example, the video processing apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the video processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0055] The video processing method provided in the embodiments of this application will be described below with reference to the accompanying drawings. The electronic device 400 that implements the video processing method of the embodiments of this application may be a terminal, a server, or a combination of the two. Therefore, the executing entity of each step will not be described again below.

[0056] The video processing method of this application embodiment will be described using a terminal (such as the first terminal 400-1) as an example. The terminal (such as the first terminal 400-1) is equipped with an emulator. The video processing method of this application can be applied to an emulator in a host environment. The emulator is used to simulate and run a video application in a virtual environment within the host environment. Here, the host environment serves as the underlying operating platform (such as a computer's Windows operating system), on which the emulator runs. This emulator is responsible for building and supporting an independent virtual environment, such as the Android Open Source Project (AOSP). When a video application running in this virtual environment responds to a playback operation, it calls a proxy decoder also located in the virtual environment. This proxy decoder does not directly perform decoding but acts as a bridge connecting the virtual environment and the host environment, intercepting the video application's decoding request and video stream data, and forwarding it to the host environment through the emulator. This allows efficient decoding to be performed using the host environment's hardware resources (i.e., a hardware decoder, such as a graphics processor). See also... Figure 3 , Figure 3 This is a first flowchart illustrating the video processing method provided in this application embodiment, which will be combined with... Figure 3 The steps shown are explained.

[0057] In step 101, decoding configuration parameters for the video application are received from the proxy decoder running in the virtual environment.

[0058] The proxy decoder is invoked by the video application in response to video playback operations; the decoding configuration parameters represent the parameter information of the video adapted to the decoding video application.

[0059] In practical applications, a video application can refer to an application installed and running in a virtual environment (such as an Android virtual machine) that has video playback capabilities. This video application can be a system-level player pre-installed in the virtual environment or a third-party application downloaded and installed by the user. Specifically, video applications include, but are not limited to: local video players, online video streaming applications, short video applications, live streaming applications, or game applications and social media applications that include video transition animations. When a user triggers a video playback operation (such as clicking the play button) in the virtual environment's interface, the video application attempts to call the standard multimedia interface provided by the virtual environment's operating system to request decoding services.

[0060] A proxy decoder is a software module or service component pre-installed in a virtual environment for video applications. Logically, it masquerades as a standard decoder within the virtual environment, existing through hooking techniques, system service replacement, or as a low-level multimedia framework plugin. When a video application initiates a decoding request through a standard interface, it actually invokes the proxy decoder. The proxy decoder does not perform the actual decoding operations within the virtual environment; rather, it acts as a relay or bridge, intercepting the video application's request and passing the relevant data to the host environment.

[0061] Decoding configuration parameters characterize the parameter information adapted to the video currently requested for playback by the decoding video application. This parameter information is essential metadata for hardware decoder initialization. Specifically, decoding configuration parameters include, but are not limited to, at least one of the following: video encoding format, indicating the compression standard of the video data, such as H.264 / AVC (video / avc), H.265 / HEVC (video / hevc), VP9, ​​etc.; video resolution information, including the width and height of the video frame; codec-specific data, which is the most critical data for initializing the hardware decoder, including header information such as the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS), used to tell the hardware decoder how to parse subsequent video frames; color format, indicating the expected color space format for the later output after decoding, such as YUV420, NV12, YV12, etc.; and other attributes, such as the video frame rate, bit rate, quality level, and rating.

[0062] In the specific implementation, a cross-environment communication channel is established between the emulator and the virtual environment (e.g., through shared memory, socket communication, or emulator-specific pipe communication mechanisms). After obtaining the aforementioned decoding configuration parameters, the proxy decoder encapsulates them into a data packet recognizable by the host environment and sends it to the emulator through this communication channel. The emulator receives this data packet, thus completing the step of "receiving decoding configuration parameters for the video application sent by the proxy decoder running in the virtual environment," thereby learning what kind of video needs to be played in the virtual environment and preparing for subsequent searching for matching hardware resources on the host environment side.

[0063] In some embodiments, Figure 4 This is a flowchart illustrating the method for obtaining decoding configuration parameters provided in an embodiment of this application. See also... Figure 4 Step 101 can be achieved through step 1011.

[0064] In step 1011, in response to receiving an initialization request sent by the proxy decoder, the initialization request is parsed to obtain the decoding configuration parameters.

[0065] The proxy decoder is created based on the video application's startup operation, and the initialization request is generated when the video application calls the proxy decoder for the first time in response to a video playback operation.

[0066] In practical applications, the startup operation of a video application refers to the action of loading and running the video application process in a virtual environment, such as when a user clicks an icon or is invoked by another process. The proxy decoder is created based on this startup operation, meaning that an instance of the proxy decoder or its associated runtime environment is synchronously established along with the initialization of the video application process. Specifically, at the operating system level of the virtual environment, by replacing the default multimedia framework interface components or utilizing dynamic injection technology, a proxy decoder is instantiated for the video application during the startup phase when initializing its underlying media components, instead of the original software decoder. This mechanism ensures that the proxy decoder can reside unnoticed within the video application's process space, being ready before the video application initiates an actual playback request. Therefore, it can directly act as a bridge between the video application and the host environment during subsequent video playback operations, intercepting and forwarding call instructions without requiring specific modifications to the video application itself.

[0067] Specifically, the emulator simulates a virtual environment. When the virtual environment responds to a startup operation for the video application, it triggers the creation of a proxy decoder within the video application's process space, putting the proxy decoder into a standby state. Subsequently, when the video application responds to a user-triggered video playback operation and attempts to call the underlying decoding component for the first time, it actually calls the already created proxy decoder, simultaneously triggering the generation of an initialization request for the proxy decoder. The proxy decoder then generates an initialization request based on the attributes of the video to be played and sends it to the emulator through a preset communication channel. The emulator detects and receives the initialization request from the proxy decoder through the communication channel. The emulator reads the header information of the initialization request data packet and identifies the data packet type as a message requesting configuration of the decoding environment. The emulator further reads the payload portion of the initialization request data packet and deserializes or parses the payload according to a predefined data structure protocol. The emulator extracts key fields from the parsed data, including the video encoding format, video width and height information, and codec-specific data. These extracted information are combined to determine the decoding configuration parameters and temporarily stored in memory for use in subsequent steps.

[0068] In some cases, the simulator can also employ a shared memory circular buffer mechanism to receive initialization requests. The simulator periodically polls or checks the shared memory status based on an interrupt mechanism; once a write pointer update is detected, it determines that new data has been received. When parsing the initialization request, the simulator first verifies the checksum of the data packet to ensure transmission integrity. Subsequently, the simulator dynamically extracts a byte sequence of the corresponding length from the data stream based on the metadata length field included in the initialization request. For codec-specific data in the decoding configuration parameters, the simulator identifies its unique start code characteristics (e.g., 0x00000001 in H.264) and separates it into independent units such as sequence parameter sets and image parameter sets. Furthermore, the simulator can parse the session identifier carried in the initialization request and assign a unique handle to the decoding configuration parameter. This handle is used to bind the configuration parameter to a specific decoding task instance in subsequent steps, thereby supporting context differentiation when multiple video applications are running simultaneously.

[0069] As an example, assume the video application is an "Online Cinema". When a user clicks the "Online Cinema" icon on the virtual environment desktop, triggering the launch, the "Online Cinema" program starts running. Simultaneously, the virtual environment's system layer automatically creates and loads a proxy decoder instance within the "Online Cinema" process. Later, when the user first clicks to play a high-definition movie within the application, the video application responds to the video playback operation and invokes the proxy decoder. The proxy decoder generates and sends an initialization request containing "MIME type: video / avc", "resolution: 1920x1080", "CSD-0: [SPS byte stream]", and "CSD-1: [PPS byte stream]". Upon receiving this initialization request, the simulator executes its parsing logic, recognizing the MIME type as video / avc, indicating H.264 format; parsing the resolution field to obtain a width of 1920 and a height of 1080; and extracting the CSD data block. The simulator ultimately stores this information in a structured manner as a decoding configuration parameter object, completing this step.

[0070] By responding to initialization requests and parsing the decoding configuration parameters, the emulator can accurately determine the decoding environment requirements of the video application before the actual video playback data stream is transmitted. This "configuration-first" mechanism allows the emulator to intervene only when the video application actually attempts to play the video (i.e., the proxy decoder is called for the first time), avoiding resource waste when the application starts but the video is not playing. Simultaneously, by directly obtaining key information such as the video encoding format and resolution by parsing the initialization request, it ensures that the subsequent hardware decoding resource allocation on the host environment side precisely matches the actual needs of the virtual environment, eliminating the risk of decoding failures or abnormal images due to inconsistent parameters, and providing an accurate initialization basis for efficient and stable hardware decoding.

[0071] See also Figure 3 The following explanation follows step 101 above.

[0072] In step 102, when a target hardware decoder with compatible decoding configuration parameters is detected in the host environment, the decoding instance of the target hardware decoder is determined in the host environment.

[0073] Here, a hardware decoder refers to a physical computing unit in the host environment that possesses hardware-accelerated video processing capabilities, or its corresponding software driver interface. This differs from software decoding methods that rely solely on the host environment's central processing unit (CPU) for general-purpose computation. In this application, the target hardware decoder can be the graphics processing unit (GPU) of the host device (i.e., the device corresponding to the host environment), or a dedicated video decoding core (Video Core / Engine) integrated within the GPU. This GPU, utilizing its highly parallel computing architecture or dedicated circuit logic, can efficiently adapt to the high-throughput video data specified in the decoding configuration parameters, supporting specific encoding formats and high-resolution rendering. Correspondingly, determining the decoding instance of the target hardware decoder in the host environment refers to calling the host environment's underlying graphics interface or media library to establish a decoding context or session object associated with the GPU (hardware decoder). This decoding instance, as a logical entity controlling the GPU, encapsulates the buffer resources allocated in the GPU's video memory for the current decoding task, the status register configuration, and the instruction stream channel, thereby establishing a connection between the simulator data and the GPU's hardware computing power.

[0074] The matching decoding configuration parameters refer to the functional attributes of the target hardware decoder, encompassing technical specifications such as video encoding format, resolution range, and color space specified in the decoding configuration parameters. During the detection process, the simulator calls the media capability query interface provided by the host environment to traverse all currently registered hardware decoder components, comparing the various multimedia types supported by each hardware decoder component with the decoding configuration parameters to determine if a matching target hardware decoder exists. A decoding instance, on the other hand, refers to a logical object or context handle within the media framework layer of the host environment, representing a specific decoding session. This decoding instance encapsulates the memory resources, status register information, and input / output buffer queues allocated by the target hardware decoder for the current task. When the simulator determines a decoding instance in the host environment, it essentially obtains a reference or pointer to this logical object, thereby establishing a control channel between the simulator's data transmission layer and the underlying physical hardware resources, ensuring that subsequent video data can be correctly routed to the target hardware decoder for processing through this decoding instance.

[0075] In some embodiments, Figure 5This is a flowchart illustrating the method for determining a decoding instance provided in an embodiment of this application. See also... Figure 5 Step 102 can be achieved through steps 1021 to 1023.

[0076] In step 1021, multiple hardware decoders available in the host environment are obtained, and a decoding instance for each hardware decoder is created.

[0077] In practical applications, the emulator scans and obtains multiple currently registered and available hardware decoders by accessing the underlying multimedia framework or driver registry of the host environment. For each identified hardware decoder, the emulator calls the initialization interface to request the establishment of a multimedia session from the host operating system, allocates the necessary memory resources and status registers, and thus successfully creates a corresponding decoding instance for each hardware decoder.

[0078] In some cases, a parallel thread pool mechanism can be used when creating decoding instances by calling the initialization interface. The simulator starts an independent initialization thread for each detected hardware decoder and calls the initialization interface concurrently. During parallel execution, if the initialization interface call fails due to a hardware failure or driver exception, the simulator catches the exception and only marks the hardware decoder as unavailable in the current thread, without interrupting the instance creation process of other hardware decoders, thereby improving the overall initialization efficiency.

[0079] In step 1022, the decoding capability information of the decoding instance of each hardware decoder is determined, and the hardware decoder is tested to see if it is compatible with the decoding configuration parameters based on the decoding capability information, and the test result is obtained.

[0080] In practical applications, the simulator calls the capability query interface for each decoding instance that is already running. This interface directly communicates with the underlying hardware driver to read the actual technical specifications of the decoding instance in the current runtime environment, including supported codec standards, resolution extremes, and quality levels, which are then used to determine the decoding capability information. The simulator performs a line-by-line matching and verification process between the video encoding format, resolution, and proprietary data in the decoding configuration parameters and the obtained decoding capability information for each decoding instance, generating a detection result indicating compatibility.

[0081] In some cases, when calling the capability query interface to determine decoding capability information, in addition to querying the static list of supported formats, it is also possible to further query the current dynamic load capacity. The capability query interface can return the current memory utilization or computing load of the hardware decoder, and the simulator incorporates this dynamic load data into the decoding capability information. During comparison, it not only verifies whether the formats match, but also combines the load requirements calculated from the decoding configuration parameters to determine whether the hardware decoder has sufficient remaining resources to handle the current task.

[0082] In step 1023, based on the detection results, a target hardware decoder that matches the decoding configuration parameters is selected from multiple hardware decoders, and the decoding instance of the target hardware decoder is retained.

[0083] In practical applications, based on the detection results, the simulator excludes all incompatible or non-compliant hardware decoders and selects one suitable hardware decoder from the remaining candidates as the target hardware decoder. The simulator retains a handle reference to the corresponding decoding instance of the target hardware decoder in memory for subsequent decoding and triggers resource reclamation logic to destroy other unselected decoding instances.

[0084] In some cases, after retaining the decoding instance of the target hardware decoder, other decoding instances that are not retained can be released using either a delayed release or immediate release strategy. If immediate release is used, the emulator calls the destruction interface to forcibly reclaim the video memory and handle resources occupied by the unselected instances. If delayed release is used, these instances are temporarily placed in a cache pool with a lifespan set. If a new decoding task with a different configuration arrives within a short period, the instances in the cache pool can be reused directly, and if they are not used after the timeout, they are physically destroyed.

[0085] As an example, if the host environment has one integrated graphics card (Intel) and one discrete graphics card (NVIDIA) installed, the emulator scans and discovers these two hardware decoders. It then calls the initialization interface to create an "Intel decoding instance" and an "NVIDIA decoding instance," respectively. Next, the emulator calls the capability query interface for each of these decoding instances. The query returns that the "Intel decoding instance" supports H.264 with a maximum resolution of 4K, and the "NVIDIA decoding instance" supports H.264 with a maximum resolution of 8K. The current decoding configuration parameters require H.264 format and a resolution of 1080P. The comparison shows that both are compatible, and the detection results are both passed. Based on the detection results (without considering priority sorting logic), the emulator can select the NVIDIA graphics card as the target hardware decoder. The emulator locks a reference to the "NVIDIA decoding instance" in its internal mapping table and simultaneously calls the release interface to destroy the "Intel decoding instance" to free up memory.

[0086] This approach, which employs a strategy of creating an instance first and then querying capabilities, has the core advantage of obtaining the most accurate runtime hardware status. Traditional methods that only query static registry entries or configuration files may result in "false positives"—drivers claiming support but the hardware actually failing due to resource exhaustion or being monopolized by other processes. Creating a decoding instance by actually calling the initialization interface effectively performs a real pre-allocation test of hardware resources, ensuring that the hardware decoders participating in subsequent detection steps are physically available. Combined with directly querying active instances via the capability query interface, this avoids the risk of inconsistencies between static configurations and dynamic environments, improving the reliability and stability of the selected target hardware decoder and guaranteeing the success rate of decoding tasks.

[0087] In some embodiments, the step 1023 of "filtering out the target hardware decoder that matches the decoding configuration parameters from multiple hardware decoders" can be implemented in the following way: when there are multiple hardware decoders that match the decoding configuration parameters, the load status of each hardware decoder that matches the decoding configuration parameters is obtained; among the multiple hardware decoders that match the decoding configuration parameters, the target hardware decoder whose load status is lower than the load threshold is filtered out.

[0088] In practical applications, after confirming the existence of multiple hardware decoders in the host environment that can adapt to the decoding configuration parameters, the simulator initiates load detection logic. For each hardware decoder that adapts to the decoding configuration parameters, the simulator calls the performance detection interface of the host operating system or the driver layer status query command. Through this interface, it reads the current runtime metrics of each hardware decoder, specifically including the real-time utilization percentage of the video decoding engine, the percentage of used video memory, and the length of the currently waiting task queue. The simulator quantifies and summarizes these real-time metric values ​​to determine the load status of that hardware decoder. Next, the simulator reads a pre-set load threshold or a load threshold generated according to the current system policy. The simulator compares the load status values ​​of each hardware decoder obtained in the first step with this load threshold. During the comparison, the simulator marks hardware decoders with load status values ​​greater than or equal to the load threshold as high-load objects and excludes them, while retaining hardware decoders with load status values ​​strictly less than the load threshold. From the retained hardware decoders, one is selected as the target hardware decoder for final use.

[0089] In some cases, a multi-dimensional weighted evaluation mechanism can be used to obtain load status, rather than a single point-in-time snapshot. The simulator continuously collects various performance data of the hardware decoder over a period of time, including instantaneous utilization, utilization variance (reflecting fluctuation stability), and core temperature data. The simulator performs weighted calculations on these data according to preset weighting coefficients to generate a comprehensive score as the load status. This method can identify hardware decoders that, although currently underutilized, are about to trigger hardware throttling protection due to excessive temperature, or have drastic and unstable load fluctuations, thus avoiding potential performance pitfalls. When selecting target hardware decoders, dynamic thresholds and bidding selection strategies can also be introduced. The load threshold does not use a fixed value but is dynamically calculated based on the video bitrate and resolution in the decoding configuration parameters (e.g., high-resolution videos correspond to more stringent low load thresholds). If, after screening, multiple hardware decoders still have load statuses below the load threshold, the load status values ​​of these candidates are further compared, and the one with the lowest load status is directly selected as the target hardware decoder to achieve optimal resource matching. If the load status of all hardware decoders is higher than the load threshold, then the fallback logic is executed to force the selection of the one with the lowest load status, or to prompt the user to switch to software decoding.

[0090] As an example, assume the host environment has integrated graphics card A and dedicated graphics card B installed, both of which have passed the previous steps and are compatible with the current decoding configuration parameters. The simulator performs the first step, finding that integrated graphics card A currently has a decoding core utilization rate of 75%, while dedicated graphics card B currently has a decoding core utilization rate of 20%. The simulator performs the second step, setting the load threshold to 60%. The simulator compares the load and finds that integrated graphics card A's load (75%) is higher than the load threshold, while dedicated graphics card B's load (20%) is lower than the load threshold. Based on this, the simulator excludes integrated graphics card A and selects dedicated graphics card B as the target hardware decoder.

[0091] This approach, building upon functional adaptation, further incorporates performance-level assessment. Its core advantage lies in achieving load balancing of hardware resources and ensuring the reliability of decoding tasks. In environments with multiple hardware components, static functional support alone cannot reflect the real-time processing capabilities of the hardware. Forcibly assigning tasks to a hardware decoder that supports the format but is already under heavy load can easily lead to decoding delays, dropped frames, or even system stuttering. By monitoring the load status in real time and using load thresholds for filtering, it ensures that the selected target hardware decoder has sufficient idle computing resources to handle the current decoding task, thereby minimizing the risk of performance bottlenecks during decoding and improving the smoothness of video playback and the overall system responsiveness.

[0092] See also Figure 3The following explanation follows step 102 above.

[0093] In step 103, when the first video stream data originating from the video application is received from the proxy decoder, the first video stream data is decoded by the decoding instance of the target hardware decoder to obtain the first decoded data.

[0094] In practical applications, the simulator receives first video stream data intercepted from the video application and transmitted by the proxy decoder via inter-process communication. This first video stream data typically consists of a sequence parameter set, an image parameter set, and compressed image frame data (such as NAL units). The simulator locks the input buffer associated with the decoding instance of the target hardware decoder and writes the first video stream data into this input buffer. Subsequently, the simulator sends a decoding commit command to the decoding instance. The decoding instance drives the underlying hardware logic of the target hardware decoder (such as a dedicated video processing core) to perform decompression operations such as entropy decoding, inverse quantization, inverse transform, and motion compensation based on the encoding format information contained in the first video stream data. After the operations are completed, the target hardware decoder fills the restored original pixel matrix data into the output buffer associated with the decoding instance. The simulator reads the pixel matrix data from this output buffer or obtains a handle object pointing to the output buffer, identifying it as the first decoded data.

[0095] In some cases, asynchronous pipelines and frame drop compensation mechanisms can be introduced during decoding via a decoding instance. When submitting the first video stream data to the decoding instance, the simulator does not use a blocking wait approach, but instead maintains an input task queue and an output callback detector. When the data transmission rate of the video application momentarily exceeds the processing rate of the target hardware decoder, the simulator checks the backlog in the input task queue. If the backlog exceeds a preset threshold, the simulator parses the type of the data frame to be decoded, actively discarding non-critical frames (such as P-frames or B-frames), and only retaining critical frames (I-frames) to submit to the decoding instance, thereby reducing hardware load and quickly catching up with timestamps. Simultaneously, when acquiring the first decoded data, the simulator also reads the decoding status metadata fed back by the hardware decoder. If a decoding error is detected (such as screen tearing or data corruption), it automatically triggers the reset logic for requesting critical frames, ensuring that the generated image data is valid and usable.

[0096] By directly driving hardware-level decoding instances to process the raw bitstream, the core advantage of this approach lies in offloading computationally intensive tasks to hardware. The decompression process of the first video bitstream data is handled by the dedicated circuitry of the target hardware decoder, avoiding heavy software decoding operations on the host device's CPU and significantly reducing system resource consumption and power consumption in the host environment. Furthermore, hardware decoding instances typically possess pipeline architectures optimized for specific video formats, enabling them to convert compressed bitstreams into usable pixel data with lower latency compared to general-purpose computing, ensuring real-time performance and smoothness in high-resolution or high-frame-rate video processing scenarios.

[0097] In some embodiments, before decoding the first video stream data through the decoding instance of the target hardware decoder, that is, after determining the decoding instance of the target hardware decoder, the simulator may also send a first prompt message to the proxy decoder, so that the proxy decoder prompts the video application to send the first video stream data to the proxy decoder based on the first prompt message, and sends the first video stream data to the simulator through the proxy decoder; wherein, the first prompt message indicates that the decoding instance of the target hardware decoder has been successfully created.

[0098] In practical applications, after the simulator completes the screening of the target hardware decoder and the creation and retention of the decoding instance, it generates a message packet containing a specific status identifier, which is identified as the first prompt message. This first prompt message clearly indicates that the underlying hardware resources are ready, that is, it indicates that the decoding instance of the target hardware decoder has been successfully created. The simulator sends the first prompt message to the proxy decoder running in the virtual environment through a pre-established cross-process communication channel. The proxy decoder receives and parses the first prompt message, confirming that the hardware decoding path has been established. Subsequently, the proxy decoder removes the data input blocking restriction for the video application, or actively calls the callback interface of the video application to notify the video application that it can start delivering data. The video application responds to this notification, begins to read and output the first video stream data to the proxy decoder, and the proxy decoder then forwards the received first video stream data to the simulator.

[0099] In some cases, the initial notification message may also include detailed configuration constraint metadata about the target hardware decoder. Besides carrying a successful creation status code, the initial notification message encapsulates parameters such as the memory alignment actually supported by the target hardware decoder, the maximum input buffer capacity, and the recommended slice size. Upon receiving this extended information, the proxy decoder dynamically adjusts its caching strategy when receiving data from the video application based on these constraint parameters, and fine-tunes the video application's configuration before prompting it to send data. For example, if the target hardware decoder requires specific memory alignment, the proxy decoder pre-allocates shared memory blocks that meet the alignment requirements for the video application to write to, thereby achieving zero-copy or fast direct memory access when forwarding to the emulator, further reducing data transmission latency.

[0100] As an example, the emulator successfully initializes the graphics card resources for decoding on the host side. The emulator constructs an integer signal with the value "INIT_SUCCESS" as the first prompt message, writes it to the shared memory circular buffer, and notifies the proxy decoder. After reading the signal, the proxy decoder changes its internal state machine to "running" and returns a relevant ready event to the video player application (video application) that called its interface. Upon receiving the event, the video player application begins writing the downloaded H.265 video stream segment as the first video stream data into the buffer provided by the proxy decoder. The proxy decoder captures this data and transmits it to the emulator for decoding via the communication channel.

[0101] This approach, through an explicit message notification mechanism, achieves precise state synchronization and flow control across heterogeneous systems in different environments. Before the hardware decoding instance is created, data transmission from the video application is effectively suppressed, preventing buffer overflows, data loss, or invalid memory usage caused by the upper-layer application prematurely sending large amounts of video stream data before the underlying hardware is ready. By sending a first notification message as a trigger signal for data transmission, the timing of data production and consumption is ensured, eliminating the risk of race conditions during startup and improving the stability and resource utilization of the decoding system during initialization.

[0102] In some embodiments, before decoding the first video stream data through the decoding instance of the target hardware decoder, the simulator may also perform format conversion on the first video stream data to obtain the first video stream data in the target data format; the target data format is a data format supported by the decoding instance of the target hardware decoder; then, the first video stream data in the target data format is decoded through the decoding instance of the target hardware decoder.

[0103] In practical applications, after receiving the first video stream data, the simulator reads the configuration specifications of the target hardware decoder to determine the input data standard that the decoding instance can recognize, i.e., the target data format. The simulator parses the header identifier and payload structure of the first video stream data, comparing the differences between the current format and the target data format. If they are inconsistent, the simulator calls its built-in stream conversion engine to perform unpacking and reassembly operations on the first video stream data. For example, it converts the length prefix-based encapsulation to a start code-based encapsulation, or adjusts the order of data packets, thereby generating first video stream data that conforms to the target data format. Subsequently, the simulator sends the converted first video stream data in the target data format to the input buffer of the decoding instance, driving the decoding instance to decompress the data according to the target data format specifications, ultimately generating the first decoded data.

[0104] In some cases, a deep reconstruction and memory adaptation mechanism based on encoded metadata can be introduced for the format conversion step. During the generation of the target data format, the simulator not only adjusts the physical encapsulation of data packets but also deeply scans the encoding layer information in the first video stream data. If the target hardware decoder requires the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) to be embedded in the bitstream, and the source data uses out-of-band transmission mode, the simulator automatically extracts the cached parameter set data and inserts it into the header of each keyframe's data packet. Simultaneously, considering hardware decoding efficiency, the conversion process also includes memory layout optimization. Based on the direct memory access characteristics of the target hardware decoder, the simulator moves the first video stream data to a specific physical contiguous memory block and pads the data tail with invalid bytes according to the hardware's required byte alignment rules (such as 128-byte alignment), constructing a target data format that satisfies both logical and physical storage requirements, thereby reducing bus overhead during subsequent hardware data reading.

[0105] As an example, the first video stream data output by the video application uses the AVCC format (common in MP4 files), where each NAL unit contains 4 bytes of length information before it, and the SPS / PPS information is stored in the avcC atom of the global header. The target hardware decoder in the host environment (such as a certain model of DSP) only supports the Annex-B format, requiring the use of "00 00 00 01" as the start code, and requiring the SPS / PPS to be transmitted with the keyframe. After receiving the data, the simulator strips the AVCC format length prefix, replaces it with the Annex-B start code, and actively inserts the SPS and PPS data segments before the frame data when a keyframe is detected, combining them into the first video stream data of the target data format, and then submits it to the DSP for decoding.

[0106] By adding a format adaptation step to the decoding link, the core advantage of this approach lies in significantly enhancing hardware compatibility and system robustness in heterogeneous environments. Low-level hardware decoders often have strict and specific constraints on the encapsulation protocols, parameter arrangements, and memory layout of input data. Directly transmitting source data can easily lead to decoding errors or hardware hangs. By uniformly converting the raw bitstream into a hardware-specific target data format, the diversity of upper-layer application data encapsulation and the differences between the underlying hardware interface specifications are effectively shielded, ensuring that hardware decoding instances always operate under standardized input conditions, thereby improving the success rate of video decoding.

[0107] In some embodiments, the first decoded data is stored in a decoded data queue, and the host environment is configured with input processing threads and output processing threads running in parallel. When the simulator receives the first video stream data originating from the video application sent by the proxy decoder, the simulator can call the input processing thread to send the first video stream data to the decoding instance of the target hardware decoder, and decode the first video stream data through the decoding instance of the target hardware decoder. After obtaining the first decoded data, the simulator can call the output processing thread to store the first decoded data in the decoded data queue.

[0108] In practical applications, the host environment starts two independent execution flows during the initialization phase, labeled as the input processing thread and the output processing thread, respectively. When the first video stream data arrives, the input processing thread obtains the memory address and length information of the data, requests the input buffer index of the decoding instance of the target hardware decoder, fills the first video stream data into the input buffer, and submits decoding instructions to the decoding instance, thereby driving the underlying hardware to start decoding operations. Simultaneously, the output processing thread is in a polling or blocked waiting state, detecting changes in the state of the decoding instance. Once the decoding instance completes the processing of one frame and generates the first decoded data, a completion signal is triggered. The output processing thread captures this completion signal, obtains the output buffer information containing the first decoded data, encapsulates it into a data object, and pushes it into a pre-allocated memory container, i.e., the decoded data queue, in a first-in, first-out order for subsequent processes to read.

[0109] In some cases, a double-buffered or circular-buffered synchronization mechanism based on a producer-consumer model is constructed. The input processing thread, acting as the producer, maintains a free buffer pool, performs preprocessing (such as NAL unit segmentation) before submitting data to the decoding instance, and uses semaphores to control the submission rate to avoid overflow. The output processing thread, acting as the consumer, performs synchronization correction of metadata (such as timestamps and color gamut information) when obtaining the first decoded data from the decoding instance. To further improve efficiency, a fine-grained mutex or lock-free queue algorithm is implemented on the decoded data queue to reduce thread context switching overhead. Simultaneously, a dynamic thread scheduling strategy is introduced to adjust the priorities of the input and output processing threads in real time based on the CPU load and video bitrate fluctuations of the current host environment. For example, when the decoded data queue is about to fill up, the priority of the input processing thread is temporarily reduced, and the priority of the output processing thread is increased, and vice versa, thereby maintaining the dynamic balance of the pipeline.

[0110] By breaking down the decoding process into two independent threads—input and output—running in parallel, the video data submission and image frame acquisition are completely decoupled. This asynchronous pipeline architecture allows the host environment to continue preparing subsequent bitstream data or processing output images while the hardware decoder performs time-consuming calculations, eliminating the waiting and idle time caused by serial processing. The input processing thread ensures that the hardware decoder always has data to decode, avoiding hardware starvation; the output processing thread ensures that decoded data is promptly transferred, avoiding hardware pipeline blockage. Their collaborative work improves the utilization of encoding and decoding resources and the overall system throughput, making it particularly suitable for high-concurrency or high-resolution video processing scenarios.

[0111] See also Figure 3 The following explanation follows step 103 above.

[0112] In step 104, the first decoded data is sent to the proxy decoder so that the first decoded data can be sent to the video application for video playback via the proxy decoder.

[0113] In some embodiments, Figure 6 This is a flowchart illustrating the method for sending the first decoded data provided in an embodiment of this application. See also... Figure 6 The "send first decoded data to the proxy decoder" in step 104 can be achieved through steps 1041 to 1044.

[0114] In step 1041, a video acquisition request originating from a video application is received from the proxy decoder.

[0115] In practical applications, the emulator continuously monitors messages from the proxy decoder via an inter-process communication channel. When a video application needs to acquire image frames for rendering or processing, it initiates a request. The proxy decoder intercepts this request and forwards it to the emulator; this is known as a video acquisition request. This video acquisition request may contain identifiers to distinguish different video stream sessions and image data attribute information that the upper-layer application expects to receive.

[0116] In step 1042, the video acquisition request is parsed to obtain the target video identifier and target color format of the target video requested by the video application.

[0117] In practical applications, after the simulator receives the video acquisition request, it deserializes and parses the payload, extracts the ID field used to uniquely identify the video stream, and determines it as the target video identifier. At the same time, it extracts the enumeration values ​​(such as RGB_8888, YV12, etc.) that represent the pixel arrangement and color space, and determines them as the target color format.

[0118] In step 1043, when the video identifier corresponding to the first decoded data is the target video identifier, the color format of the first decoded data is determined.

[0119] In practical applications, the simulator searches for the first decoded data that matches the target video identifier in its internally maintained queue of decoded frames (i.e., the decoded data queue). Once found, the simulator reads the metadata (such as buffer descriptors) that accompanied the first decoded data when the target hardware decoder outputs it, identifies its original pixel encoding method (usually a native hardware format such as NV12 or P010), and thus determines the color format of the first decoded data.

[0120] In step 1044, when the color format of the first decoded data is different from the target color format, the color format of the first decoded data is adjusted to the target color format, and the first decoded data in the target color format is sent to the proxy decoder.

[0121] In practical applications, the simulator compares the color format of the first decoded data with the target color format. If they are inconsistent, the simulator calls its internally integrated image processing algorithm library to execute color space conversion logic. For example, the CPU calculates and maps the luminance component Y and chrominance components U / V to red, green, and blue pixel values ​​according to a standard conversion formula, or adjusts the memory layout order of the components to generate a new data block that conforms to the target color format (i.e., the first decoded data in the target color format). Finally, the simulator writes this adjusted data block to the shared transmission channel and sends it to the proxy decoder.

[0122] In some cases, context-aware intelligent caching and parallel computing optimization mechanisms can be introduced during color format adjustment and transmission. After parsing the target color format, the simulator not only performs a single conversion but also establishes a stream-level format mapping table. For subsequent frames of the same continuously arriving video stream, the simulator pre-allocates a dedicated buffer pool that meets the memory alignment requirements of the target color format. During format adjustment, the simulator utilizes the host environment's CPU's multi-instruction stream single-data stream instruction set (such as AVX or NEON) to vectorize and accelerate the pixel conversion algorithm, processing color matrix operations for multiple pixels at once, rather than performing serial calculations pixel by pixel. Simultaneously, the simulator detects conversion time; if it detects excessive CPU load, it can dynamically reduce conversion accuracy (e.g., using a fast approximation algorithm) to prioritize real-time performance. When transmitting data, the simulator packages the converted data block with auxiliary information such as timestamps and frame types, synchronizes it to the memory area visible to the proxy decoder via direct memory access, and triggers an interrupt signal to minimize latency caused by data copying.

[0123] As an example, a user is previewing video footage using a video editing app (i.e., a video application). The video application requests that the footage be displayed in ARGB format, corresponding to the video stream ID Stream_01. The proxy decoder sends a video retrieval request containing "Stream_01" and the "ARGB" requirement. After parsing, the emulator checks the buffer output by the current hardware decoder and finds a frame of decoded data belonging to Stream_01. However, the original data decoded by the graphics card is in half-plane NV21 format. The emulator determines that NV21 is different from ARGB format and immediately initiates a software conversion process. It reads the Y-plane and interleaved VU-plane data of NV21, calculates the corresponding ARGB value using a formula, and fills it into a new memory block. After the conversion is complete, the emulator sends this ARGB data to the proxy decoder. The video application receives it and can directly render and play it as a texture.

[0124] By implementing an on-demand format adaptation strategy on the simulator side, the underlying hardware capabilities and upper-layer application requirements are efficiently decoupled. Hardware decoders typically output only specific YUV formats to optimize bandwidth, while upper-layer video applications often require RGB formats for easier rendering or processing. By parsing requests and automatically performing format adjustments, the simulator shields the output differences of the underlying hardware, eliminating the need for video applications to modify their rendering pipelines to adapt to different hardware decoders. This mechanism ensures that image data always conforms to its expected standards when delivered to the application layer, improving format compatibility and avoiding display anomalies or application crashes caused by format mismatches.

[0125] In some embodiments, the step 1044 of "adjusting the color format of the first decoded data to the target color format" can be implemented by: creating a shader component in the host environment; binding the first decoded data to the input texture of the shader component; and calling the decoding instance of the target hardware decoder to run the shader component, so as to perform color space conversion on the input texture through the shader component and generate the first decoded data that conforms to the target color format.

[0126] In practical applications, the simulator initializes the rendering pipeline in the graphics rendering subsystem of the host environment (such as the OpenGL or Vulkan context), compiles and links fragment shaders or computational shaders that contain color space conversion algorithms (such as matrix operation logic for YUV to RGB), constructs an executable graphics card instruction set object, and identifies it as a shader component.

[0127] Next, the simulator obtains the underlying video memory handle where the first decoded data output by the target hardware decoder is located. It then encapsulates this handle via the graphics interface extension protocol and associates it with the texture sampler unit of the shader component, establishing it as the input texture. Subsequently, the simulator configures the rendering target frame buffer to match the parameter requirements of the target color format and submits drawing or scheduling instructions to the graphics command queue associated with the target hardware decoder hardware resources. This triggers the graphics processing unit associated with the decoding instance of the target hardware decoder to begin executing the logic of the shader component. The graphics processing unit samples each pixel in the input texture according to the parallel algorithm defined by the shader component, reads the original color component values, performs floating-point matrix multiplication and clamping operations, and writes the calculation results to the corresponding location in the target buffer in real time. Once all pixels have been processed, the data in the target buffer becomes the first decoded data that has completed format conversion and conforms to the target color format.

[0128] In some cases, during the binding and runtime phases, a zero-copy-based texture mapping and fence synchronization mechanism can be implemented. Without any data transfer from VRAM to memory, the hardware buffer of the first decoded data is directly mapped to an external texture object that the shader component can directly read using a graphics interface (such as EGLImage or VK_EXTERNAL_MEMORY). Simultaneously, hardware synchronization fence signals are inserted between the output data of the decoded instance and the start of processing by the shader component, and between the completion of processing by the shader component and subsequent transmission. The shader component automatically executes after waiting for the decoding completion signal in the GPU command stream, without CPU polling, achieving purely hardware-level pipelined connection. Furthermore, to meet the needs of different target color formats, multiple shader variants are pre-compiled and cached in VRAM, dynamically switching at runtime based on requests to avoid redundant compilation overhead.

[0129] As an example, the host environment is based on an Android emulator, running an embedded open graphics library graphics stack at the underlying level. The emulator compiles a shading language fragment shader (shader component) to convert NV21 format to RGB_8888 format. The target hardware decoder decodes a frame (first decoded data) and stores it in the graphics buffer. The emulator calls the image creation extension interface to wrap the graphics buffer as an image object and then binds it to an external texture target (input texture). Next, the emulator binds an empty RGB_8888 texture as an attachment to the frame buffer and calls the array drawing command. The host graphics processor then runs the shader, reads the NV21 texture data, calculates the red, green, and blue color values, and fills them into the RGB_8888 texture, finally obtaining the converted first decoded data.

[0130] By utilizing the high parallel computing power of the graphics processing unit (GPU, hardware decoder) to replace the general-purpose processor (CPU) in performing high-load image processing tasks, the throughput and real-time performance of color space conversion are significantly improved. By completing the "decoding-conversion" process in a closed loop within the video memory, bus congestion and latency caused by the back-and-forth copying of high-bandwidth video data between video memory and system memory are avoided. This reduces the overall system power consumption and CPU utilization in the host environment, ensuring smooth playback in high-resolution, high-frame-rate video scenarios.

[0131] In this embodiment, when the emulator simulates a video application in a virtual environment within a host environment, and receives decoding configuration parameters for the video application from a proxy decoder (called by the video application in response to video playback operations), if a target hardware decoder matching the decoding configuration parameters is detected in the host environment, the emulator determines the decoding instance of the target hardware decoder in the host environment. Upon receiving the first video stream data originating from the video application from the proxy decoder, the emulator decodes the first video stream data using the decoding instance of the target hardware decoder to obtain the first decoded data. Finally, the emulator returns the first decoded data to the proxy decoder so that the video application can play video based on the first decoded data. This method enables video decoding to be performed directly through the decoding instance of the hardware decoder in the host environment when the video application in the virtual environment plays video, reducing the resource consumption of the video application's central processing unit and thus improving video decoding efficiency, thereby enhancing the smoothness of video playback in the video application.

[0132] In some embodiments, after obtaining the target video identifier of the target video requested by the video application in step 1042, when the video identifier corresponding to the first decoded data is different from the target video identifier, the simulator may also send a second prompt message to the proxy decoder to prompt the proxy decoder to send the second video stream data of the target video to the simulator; wherein, the second prompt message indicates that there is no decoded data of the target video, and the second video stream data corresponds to the target video identifier; when the second video stream data is received, the second video stream data is decoded by the decoding instance of the target hardware decoder to obtain the second decoded data, and the second decoded data is sent to the proxy decoder to send the second decoded data to the video application for video playback through the proxy decoder.

[0133] In practical applications, after parsing the target video identifier from the video acquisition request, the simulator compares it with the video identifier associated with the first decoded data already ready in the current buffer using either a string or hash value. If they do not match, the simulator determines that a video stream switch or data asynchrony has occurred, and immediately terminates the transmission process of the current first decoded data. It then generates a data packet containing a specific status code (such as a data missing marker) and the target video identifier, constructing a second prompt message. The simulator sends the second prompt message to the proxy decoder through a preset inter-process communication feedback channel, explicitly informing the proxy decoder that the target video decoding resource is currently missing and requesting the supplementation of the corresponding video stream data.

[0134] After the proxy decoder responds to the second prompt and sends back the compressed data of the target video, the simulator captures this data through the data receiving interface and identifies it as the second video stream data. The simulator immediately invokes the decoding instance of the target hardware decoder and writes the second video stream data into the input circular buffer of the decoding instance. The decoding instance initiates hardware acceleration logic to decompress the data and outputs the corresponding original image frame, i.e., the second decoded data. The simulator then encapsulates the second decoded data into a shared memory block and notifies the proxy decoder to read it through a communication mechanism, thereby completing the data delivery to the video application.

[0135] In some cases, a fast context switching and prefetching mechanism based on state prediction can be introduced for sending the second prompt message and subsequent processing steps. When a video identifier inconsistency is determined, the simulator not only sends the second prompt message but also immediately suspends the current decoding instance and performs a "soft reset" operation, releasing texture resources in video memory. Simultaneously, it queries the historical records based on the target video identifier; if a cache configuration for that video exists, it preloads it into the decoder register. When sending the second prompt message, the simulator appends a precise timestamp request to the message payload, instructing the proxy decoder to start sending the second video stream data from a specific time point to ensure frame-level continuity of the video. When receiving and decoding the second video stream data, the simulator adopts a "Fast-Forward" mode, increasing the clock frequency or priority of the hardware decoder to prioritize processing this newly arrived data. The generated second decoded data is directly mapped to the proxy decoder's read address space, skipping the regular queue waiting process to compensate for the latency caused by video switching in the shortest possible time.

[0136] As an example, a user swipes from video A (ID: 1001) to video B (ID: 1002) in a short video application (i.e., a video application). The video application requests to display the frame of video B. The simulator checks the currently decoded frame and finds it to be a frame of video A (ID: 1001), determining a mismatch. The simulator sends a second prompt message in JSON format, {"status":"MISSING","target_id":"1002"}, to the proxy decoder. Upon receiving this message, the proxy decoder retrieves the H.264 bitstream of video B (second video bitstream data) from the application layer and sends it to the simulator. The simulator receives the second video bitstream data, drives the decoding instance of the hardware decoder to decode, obtains the YUV data of video B (second decoded data), and sends it to the proxy decoder. The user's screen then displays the frame of video B.

[0137] The above solution addresses the data asynchrony issue during video stream switching in heterogeneous environments by establishing a proactive anomaly feedback and on-demand request closed-loop mechanism. In scenarios where video applications frequently switch playback content, the simulator can promptly detect misalignments between upper-layer requirements and lower-layer resources, proactively driving data source updates via a secondary prompt, rather than passively waiting or transmitting incorrect images. This mechanism ensures that the hardware decoder always processes the video content that the user is truly interested in, avoiding invalid rendering of old data, significantly reducing black screen duration or the probability of screen tearing during video switching, and guaranteeing the smoothness and accuracy of the user's visual experience.

[0138] In some embodiments, after obtaining the first decoded data, the simulator may also call a pre-trained feature extraction model to extract features from the first decoded data to obtain decoded features; perform enhancement processing on the first decoded data based on the decoded features to obtain enhanced decoded data; and send the enhanced decoded data to the proxy decoder so that the enhanced decoded data can be sent to the video application for video playback through the proxy decoder.

[0139] In practical applications, after obtaining the first decoded data, the host environment loads and initializes a pre-trained neural network model, i.e., the feature extraction model. The host environment converts the first decoded data from image format to tensor format required for model computation and inputs it into the feature extraction model. The feature extraction model performs multi-layer convolution and activation operations on the input data to extract a deep abstract representation containing image texture, edges, and high-frequency information, thus obtaining the decoded features. Subsequently, a specially designed super-resolution reconstruction network or image enhancement algorithm module is used to perform non-linear mapping and pixel reconstruction based on the decoded features. This process fills in lost high-frequency details and performs upsampling operations to generate images with higher resolution or clearer image quality, thus obtaining the enhanced decoded data. Finally, the host environment writes the enhanced decoded data into a shared memory area with the virtual environment and notifies the proxy decoder through a communication mechanism. The proxy decoder reads the data from the shared memory and delivers it to the video application, thereby completing the rendering and playback of high-quality video.

[0140] In some cases, a parallel processing architecture based on heterogeneous computing can be constructed for the model inference stage. The feature extraction model and enhancement processing algorithm are deployed on dedicated AI acceleration hardware in the host environment, achieving hardware-level decoupling and parallel execution from the decoding tasks of general-purpose processors. Model compression techniques, such as network pruning and INT8 quantization, are employed to significantly reduce computational load and memory consumption while maintaining the accuracy of the enhancement effect. Simultaneously, a content-based adaptive enhancement strategy is introduced, performing region of interest detection (such as face or subtitle recognition) before processing the first decoded data. High-complexity feature extraction and enhancement processing are performed only on the identified core regions, while low-computation traditional interpolation algorithms are used for background regions. This significantly improves the image quality of key visual areas while minimizing the overall system latency, meeting the timeliness requirements of real-time video stream processing.

[0141] As an example, the simulator runs an outdated video application, playing a video stream with a resolution of only 360p. The host environment intercepts decoded 360p image frames in real time (first decoded data) and sends them to a super-resolution generative adversarial network model (feature extraction model) running in the background. The model extracts the texture features of the image (decoded features) and reconstructs a 1080p image frame with rich texture details and sharp edges (enhanced decoded data). This 1080p image frame is then sent back to the proxy decoder inside the simulator. The final user sees a high-definition image reconstructed in real time by the model on the video application interface, instead of the original blurry image.

[0142] This approach fully leverages the powerful computing capabilities and model acceleration hardware advantages typically found in host environments, overcoming bottlenecks caused by low-quality original video sources or limited network bandwidth, and achieving real-time on-device upgrades of video quality. By intelligently restoring image details through deep learning technology, it significantly improves video resolution and clarity, providing users with a superior visual experience that surpasses the limitations of the original bitstream. Furthermore, offloading the high-load image enhancement task to the host environment effectively avoids resource contention within the virtual environment, ensuring the smoothness and stability of the video application's interaction.

[0143] In some embodiments, Figure 7 This is a schematic diagram of the second process of the video processing method provided in the embodiments of this application. See also: Figure 7 After receiving the decoding configuration parameters for the video application sent by the proxy decoder running in the virtual environment in step 101, video processing can also be performed through steps 105 to 107.

[0144] In step 105, when it is detected that there is no hardware decoder in the host environment that matches the decoding configuration parameters, or when the total load of all hardware decoders in the host environment exceeds the load threshold, a decoding instance of the software decoder is created in the host environment.

[0145] In practical applications, after receiving the decoding configuration parameters, the host environment first queries its internally maintained hardware capability list, comparing the encoding format, resolution, and color depth requirements in the decoding configuration parameters. If there is no matching item in the list, or if a matching hardware decoder exists but the system status detection module finds that the current concurrent sessions or video memory usage of all hardware decoders have exceeded the preset load threshold, the host environment determines that it cannot use the hardware decoder resources. At this point, the host environment loads a software decoding library based on a general-purpose processor instruction set, allocates system memory resources according to the decoding configuration parameters, initializes the context environment, and constructs a decoding instance of the software decoder.

[0146] In step 106, when the first video stream data originating from the video application is received from the proxy decoder, the first video stream data is decoded by the decoding instance of the software decoder to obtain the third decoded data.

[0147] In practical applications, after the software decoder's decoding instance is created, the host environment receives the first video stream data from the video application via a communication channel and stores it in the system memory buffer. Subsequently, it calls the data input interface of the software decoder's decoding instance to send the first video stream data into the decoding pipeline. The host environment's general-purpose processor executes a decompression algorithm, performing entropy decoding, inverse quantization, and inverse transform operations on the first video stream data, ultimately outputting the original image frame, which is the third decoded data.

[0148] In step 107, third decoded data is sent to the proxy decoder so that the third decoded data can be sent to the video application for video playback via the proxy decoder.

[0149] In practical applications, after obtaining the third-party decoded data, the host environment writes it to the shared memory area established with the proxy decoder and generates a notification message containing the data address offset and timestamp. The host environment sends this message to the proxy decoder. Upon receiving the notification, the proxy decoder reads the data from the shared memory and delivers it to the video application, thereby completing the rendering update of the video playback screen.

[0150] In some cases, during the resource detection and instance creation phases, multi-dimensional intelligent decision-making and dynamic degradation strategies are implemented. This involves not only detecting the utilization rate of the hardware decoder but also comprehensively evaluating the host environment's CPU load, thermal design power margin, and memory bandwidth. If software decoding is deemed necessary, the optimal thread affinity and parallelism parameters for the software decoder instance are automatically configured based on the host environment's CPU core topology (e.g., big.LITTLE architecture), and single-instruction multiple-data (SIMD) optimization options are enabled to maximize software decoding efficiency. Furthermore, if switching to software decoding is necessary due to excessive load, the configuration of the decoding instance can be dynamically adjusted, such as enabling low-latency mode or appropriately reducing deblocking filtering intensity, to balance image quality and performance.

[0151] During the decoding and transmission phases, a zero-copy data transfer mechanism can be employed. The software decoder directly points the target buffer of the decoded output to the shared memory block pre-negotiated with the proxy decoder, avoiding a secondary copy from the decoder's private heap memory to the shared memory. Furthermore, considering that the third-party decoded data is typically in a flat YUV format, CPU vectorized instructions are used to quickly calculate and fill the span and padding data before transmission, ensuring that the data format strictly conforms to the rendering requirements of the video application and preventing screen tearing or misalignment due to format incompatibility.

[0152] As an example, the Android emulator attempts to simulate playing a VP9-encoded 4K video. The emulator checks the host environment's graphics card driver and finds that it does not support VP9 hardware decoding, or that the current graphics card is busy rendering a large 3D game, with the video engine load reaching 95%. The emulator then decides to use CPU software decoding, initializing the FFmpeg decoding context (the decoding instance of the software decoder). After receiving the VP9 bitstream (the first video bitstream data), the emulator uses the CPU's AVX2 instruction set to decode it, generating a YUV420P format image (the third decoded data). The emulator directly writes this third decoded data to a shared memory file named shm_video_0 and notifies the proxy decoder within the Android system to read and display it via a communication channel.

[0153] This approach constructs a highly available dual-mode decoding architecture, improving system compatibility and stability. By seamlessly switching to software processing when hardware resources are unsupported or overloaded, it ensures the continuity of video services in heterogeneous or high-load environments, effectively avoiding playback failures or application crashes caused by hardware bottlenecks. Simultaneously, this mechanism achieves dynamic load balancing of host computing resources, rationally allocating task load between the general-purpose processor and the graphics processor, guaranteeing the overall system response speed and smoothness in complex multi-tasking scenarios.

[0154] In some embodiments, during the decoding of the first video stream data by the decoding instance of the target hardware decoder in step 103, the simulator can also detect the running status of the decoding instance of the target hardware decoder in real time; when the running status indicates that the decoding instance of the target hardware decoder has encountered a decoding error, the decoding instance of the target hardware decoder is deregistered; a decoding instance of the software decoder is created in the host environment so as to decode the first video stream data by the decoding instance of the software decoder.

[0155] In practical applications, during the decoding task execution of the target hardware decoder's decoding instance, the host environment starts an independent detection coroutine or utilizes the underlying driver's callback mechanism to periodically query the health indicators of the target hardware decoder's decoding instance. Specifically, whenever an input buffer is submitted to the target hardware decoder's decoding instance or an output buffer is requested, the host environment verifies the return value of the interface call, the hardware interrupt signal, and the execution time. If the return value contains an error code, a completion signal is not received within the preset time window, or the hardware device handle becomes invalid, the host environment determines that the current running state is abnormal. Once a decoding abnormality is confirmed, the host environment immediately triggers the abnormality handling process. First, it stops sending new data packets to the target hardware decoder's decoding instance and attempts to lock the currently unprocessed data queue. Then, it calls the hardware abstraction layer's destruction interface to forcibly release the video memory resources, texture handles, and hardware context bound to the decoding instance, removing it from the active device list to complete the deregistration operation and prevent the abnormal state from further affecting system stability. Next, the host environment loads a pre-built software decoding library from memory and initializes and constructs the software decoder's decoding instance based on the original video encoding parameters (such as resolution and encoding format). The host environment retrieves previously stored or backtracked unprocessed first video stream data and redirects it to the decoding instance of the software decoder. The software decoder utilizes the computing power of the general-purpose processor (CPU) to continue the decompression operation and take over subsequent video processing tasks.

[0156] In some cases, a full-link state preservation and breakpoint resumption mechanism can be built for anomaly detection and handover recovery. During the real-time detection phase, not only explicit error codes are detected, but heuristic anomaly analysis is also introduced. The decoded output image is sampled and verified (e.g., detecting pure green screens, distorted images, or severe macroblock errors), and the jitter of the decoding frame rate is statistically analyzed. If consecutive frame drops exceed a threshold, it is considered a latent anomaly. During the deregistration and creation handover phase, a "keyframe backtracking and fast-forward catching-up" strategy is implemented. Due to the inter-frame dependency of video encoding, directly decoding from the point of error may cause distorted images. Therefore, the host environment searches backward in the buffer for the most recent real-time decoding refresh frame or keyframe, and sends that keyframe and subsequent data to the newly created software decoder instance. The software decoder decodes in the background at the highest speed but does not output images until the timestamp of the decoded frame matches the playback progress at the time of the anomaly, at which point normal output and display are restored. This method ensures strict temporal and pixel correctness of the switched image content, achieving a seamless visual transition.

[0157] This approach endows the host environment with robust fault tolerance and high availability, resolving the pain point of video playback crashes caused by complex and ever-changing hardware environments (such as driver bugs, resource preemption, and hardware compatibility issues). By dynamically detecting faults at runtime and automatically downgrading to a software solution, the host environment can "operate despite damage" even if the underlying hardware fails, ensuring the continuity of video services. This adaptive failover mechanism improves the user experience, avoids the severe consequences of a single hardware failure causing the entire application to crash or display a black screen, and ensures the robust operation of services under various extreme environments.

[0158] In some embodiments, when the video application is closed, a shutdown prompt message is generated and sent to the emulator via a proxy decoder. Correspondingly, when the emulator receives the shutdown prompt message sent by the proxy decoder for the video application, it deregisters the decoding instance of the target hardware decoder. When the deregistration of the decoding instance of the target hardware decoder is completed, a third prompt message is sent to the proxy decoder to deregister the proxy decoder. The third prompt message indicates that the deregistration of the decoding instance of the target hardware decoder was successful.

[0159] In practical applications, the host environment continuously monitors control commands from the virtual environment through a pre-established communication channel. Upon receiving a close prompt message containing a specific session identifier from the proxy decoder, the host environment immediately suspends the data receiving thread for that session and locks the corresponding target hardware decoder's decoding instance in the resource management list based on the identifier. Subsequently, the host environment calls the interface of the underlying hardware driver to perform a context destruction operation, reclaiming the video memory buffer, handle, and related computing resources occupied by the instance, thereby completing the deregistration of the target hardware decoder's decoding instance. After the underlying driver returns a status code indicating successful resource release, the host environment generates a confirmation message, i.e., the third prompt message. This third prompt message contains a status flag indicating successful operation and the corresponding session identifier. The emulator in the host environment sends the third prompt message back to the proxy decoder in the virtual environment through the communication channel. Upon receiving this third prompt message and confirming that the host-side resources have been safely released, the proxy decoder immediately triggers its own destruction logic, releasing the memory stack and system service connections it occupies in the virtual environment, thereby completing the deregistration of the proxy decoder.

[0160] In some cases, when the emulator receives a shutdown prompt, it can also encapsulate the deregistration task as an asynchronous event and submit it to a dedicated thread pool to avoid blocking the main communication thread. When deregistering the decoding instance of the target hardware decoder, a three-level exit strategy of "drain-stop-destroy" is adopted: first, data filling into the input buffer is stopped, and the hardware decoding pipeline finishes processing the remaining instructions (draining); then, a stop instruction is sent; and finally, the destruction interface is called. Simultaneously, a watchdog mechanism is introduced to detect the deregistration process. If the hardware driver does not respond to the release request within a specified time (e.g., a deadlock occurs), forced reclamation logic is triggered, directly stripping the relevant handle resources at the operating system level. Furthermore, before sending the third prompt, the emulator will also synchronously clean up the shared memory blocks and synchronization semaphores bound to the decoding session to ensure that cross-environment resource mapping relationships are released, preventing memory leaks.

[0161] As an example, a user closes a video player (video application) that is playing a high-definition movie within the emulator. The player process sends a disconnect request (close notification) to the proxy decoder, which in turn sends a close notification to the emulator. Upon receiving the close notification, the emulator in the host environment calls the graphics card interface to release the corresponding hardware decoding handle (unregister the decoding instance of the target hardware decoder) and release the occupied video memory. After the operation is complete, the host service program sends a command "SESSION_END_OK" (third notification) to the emulator, which then sends a third notification to the proxy decoder. Upon receiving this third notification, the proxy decoder confirms that the hardware resources have been reclaimed, and then releases its own Java object and JNI reference, ending the video session.

[0162] Through the aforementioned method and a rigorous closed-loop handshake process, precise synchronization and orderly management of resource lifecycles between the host and virtual environments are achieved. This ensures that scarce hardware decoding resources are promptly and completely reclaimed when services terminate, effectively preventing system performance degradation and subsequent startup failures caused by resource leaks or residues. Simultaneously, this "first-end release, second-end confirmation" exit mechanism avoids illegal memory access or dangling pointer crashes caused by unilateral state inconsistencies, enhancing the stability and robustness of cross-environment multimedia systems.

[0163] In some embodiments, the host environment is configured with shared memory shared with the virtual environment; the "sending the first decoded data to the proxy decoder" in step 104 can be implemented in the following way: storing the first decoded data in the shared memory and generating a corresponding data index identifier; sending the data index identifier to the proxy decoder so that the proxy decoder reads the first decoded data from the shared memory according to the data index identifier.

[0164] In practical applications, the simulator first allocates a shared memory block of a preset size in the host environment's memory heap and maps its handle or memory address to the host process's address space. When the target hardware decoder's decoding instance completes the decoding operation and outputs the raw image data (i.e., the first decoded data), the simulator uses a memory copy instruction to copy the first decoded data, temporarily stored in the target hardware decoder's private output buffer (such as a graphics buffer), to a specified offset location in the shared memory. If the target hardware decoder supports direct memory address output, the shared memory address can also be used directly as the decoding output target during the decoding configuration stage, thus omitting one copy. After the data writing is completed, a unique data index identifier for the frame data is generated based on the offset of the current write position, the data length, and the frame sequence number of the first decoded data in the timing sequence.

[0165] Subsequently, the emulator, utilizing the pre-established communication channel between the host and virtual environments, encapsulates a message packet containing a data index identifier and sends it to the proxy decoder. This message packet contains only control information and does not include the heavy raw image data (i.e., the first decoded data). Upon receiving the data index identifier, the proxy decoder parses out the offset and data length. Using a pre-acquired shared memory file descriptor or mapping base address, combined with the parsed offset, the proxy decoder calculates the actual memory address of the first decoded data in the virtual environment's address space. Then, the proxy decoder directly reads the data from that memory address segment in shared memory via pointer access and encapsulates it into a video frame object recognizable by the video application.

[0166] In this way, shared memory technology builds an efficient data transmission bridge between the host environment and the virtual environment. Technically, this method significantly reduces latency when transmitting large amounts of video data across environments because shared memory allows different processes or environments to directly access the same physical memory, avoiding the multiple data copying and serialization operations between kernel and user modes required in traditional inter-process communication when transmitting large amounts of data. This mechanism greatly frees up the computing resources of the host environment's central processing unit, allowing it to focus on video decoding logic, while also reducing memory bandwidth usage. This ensures that in high-resolution or high-frame-rate video playback scenarios, the data transmission rate can meet the demands of real-time playback, avoiding stuttering or audio-visual desynchronization caused by data transmission congestion.

[0167] In some embodiments, if the first decoded data is in a graphics texture format, the above step "store the first decoded data in shared memory" can be implemented in the following ways: store the first decoded data in the graphics buffer of the target hardware decoder; map the graphics buffer to shared memory; or, transfer the first decoded data from the graphics buffer to shared memory through the decoding instance of the target hardware decoder.

[0168] In practical applications, if the first decoded data is in a graphics texture format, the target hardware decoder will temporarily store the first decoded data in its private output buffer (i.e., graphics buffer) when outputting the first decoded data. In this case, the first decoded data can be written to shared memory using any of the following methods.

[0169] Method 1: Map the graphics buffer storing the first decoded data to shared memory.

[0170] Specifically, during the decoding process, the target hardware decoder instance outputs the decoded image data (i.e., the first decoded data) to a dedicated private graphics buffer managed by the kernel. The emulator can obtain the underlying file descriptor or memory handle of this graphics buffer and, using the cross-process memory sharing interface provided by the operating system kernel, directly map the physical memory page corresponding to the file descriptor to the shared memory address range pre-negotiated between the host environment and the virtual environment, making the graphics buffer logically part of the shared memory.

[0171] Method 2: Transfer the first decoded data from the graphics buffer to shared memory through the decoding instance of the target hardware decoder.

[0172] Specifically, the target hardware decoder's decoding instance stores the first decoded data in a graphics texture format in the graphics buffer. The simulator calls the hardware acceleration interface (such as the GPU's direct memory access channel or fast readback instruction) associated with the target hardware decoder's decoding instance to control the underlying hardware to directly read the first decoded data located in the graphics buffer of the video memory and copy it into shared memory.

[0173] The above approach provides an efficient data transfer path specifically tailored to the unique characteristics of graphics texture formats. Technically, by mapping the graphics buffer, the limitation of inter-process memory isolation is broken, avoiding redundant copying of high-resolution image data between user space and kernel space by the CPU, significantly reducing system bus bandwidth usage and CPU load. Furthermore, hardware transmission via decoding instances fully utilizes the high-speed direct memory access capabilities of the graphics processor or dedicated hardware unit, resolving the bottleneck of low efficiency when the CPU directly reads video memory. This ensures that ultra-high-definition formats such as 4K and 8K, or high frame rate video data, can be delivered across environments with extremely low latency, maintaining smooth video playback.

[0174] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario. First, the terms used in the embodiments of this application will be explained, including: 1) Mobile emulator: A mobile emulator is a software tool used to create a mobile device on a first terminal (such as...). Figure 1 The first terminal 400-1 in the system (specifically, a laptop computer, desktop computer, etc.) simulates a second terminal (such as... Figure 1 The second terminal 400-2 (such as a mobile device) provides a runtime environment that allows users to experience mobile applications on a first terminal without actually owning the device. This environment is primarily divided into application market programs, mobile emulator Host, and mobile emulator Guest. As an example, the video processing method of this application can be used in a scenario where a user runs an Android system (corresponding to a virtual environment) on a Windows computer (i.e., the first terminal, corresponding to the host environment), downloads Android applications, installs them, and opens them for normal use.

[0175] 2) The application market program is a software program that executes the video processing method of this application. Users can search for and download software, and manage the running status of the mobile phone emulator Host and Guest ends in this application market program.

[0176] 3) The host side of the mobile phone emulator refers to the mobile phone emulator software itself running on the first terminal, as well as the hardware abstraction layer, operating system, and user interface components that accompany it. The host side is responsible for creating and managing the virtual mobile phone environment (i.e., the guest side, or virtual environment) and provides a series of tools and interfaces so that users can interact with the virtual mobile phone environment. The host side described in this application can be a computer software implemented based on the Windows platform, hereinafter referred to as the host side.

[0177] 4) The Guest client of the mobile emulator can refer to the virtual mobile environment (i.e., virtual environment) created by the mobile emulator software on the first terminal (Windows runtime environment). In this virtual mobile environment, users can install and run mobile applications, perform various operations and experiences, just like using a real mobile phone. The Guest client described in this application is an Android system implemented based on the Oracle VM VirtualBox (vbox) solution. Its form is a VDI file, which conforms to the VDI file format specification. The Host client loads this VDI file to load and run it. Hereinafter referred to as the Guest client.

[0178] 5) The MediaCodec interface is a low-level audio and video codec interface provided by the Android system. It acts like a pipeline that "processes" raw audio and video into a standard format. Providing raw data or compressed streams to this interface results in the output of compressed data that can be directly played or transmitted, or the decoding of compressed data into renderable raw frames. When using it, a codec is first created and configured (setting resolution, bitrate, etc.), then started, entering a loop of "empty input buffer → fill data into the queue → remove output buffer → consume or render → release" until the EOS is sent. It supports both synchronous and asynchronous usage and is often used in conjunction with MediaExtractor, MediaMuxer, Surface, and AudioTrack to achieve playback, recording, transcoding, and streaming. Simply put, it is responsible for efficiently converting content into a standard bitstream, or restoring the bitstream to displayable or playable content.

[0179] 6) C2 Decoder: The C2 decoder is a decoder component implemented based on the Codec2 framework in the Android system. It belongs to the Android multimedia codec architecture. Its core responsibility is to receive compressed video or audio bitstreams, parse and decode them according to specific media formats, and output the decoded raw data to the rendering or post-processing module. It adopts a modular and loosely coupled design, supports both hardware and software implementation paths, and can access dedicated hardware acceleration resources through third-party plugins (such as Vendor plugins). It can also fall back to pure software decoding when hardware support is lacking, thus providing unified and efficient media decoding capabilities for upper-layer interfaces or applications while balancing performance and compatibility.

[0180] 7) The Direct3D 11 Video Acceleration (D3D11VA) interface is a video acceleration interface on the Windows platform based on the Direct3D 11 API. It allows applications to efficiently decode compressed video formats (such as H.264, HEVC, and VP9) using the graphics card's hardware decoding capabilities. Implemented in the D3D11 environment through the DirectX Video Acceleration (DXVA) specification, it offloads video decoding tasks to the graphics processor, significantly reducing the CPU load and improving playback smoothness and power consumption. D3D11VA provides a unified device context and resource management mechanism, seamlessly integrating with the D3D11 graphics pipeline, facilitating zero-copy video rendering and post-processing in desktop applications, game engines, or media players.

[0181] With the rapid development of the mobile application ecosystem, Android emulators (such as phone emulators) have become an important tool for developers, gamers, and enterprise users to run Android applications on their primary terminals such as computers or PCs. Currently, most mainstream Android emulator products use software decoding for video playback, meaning that the compressed video stream is decoded on the guest side via the CPU. This decoding consumes CPU resources, and its speed is limited by CPU capabilities, easily leading to video stuttering and frame rate drops.

[0182] In related technologies, Android emulators generally adopt a guest-side software decoding scheme, where video decoding tasks are entirely handled by a virtual central processing unit (CPU) within the emulator. However, these virtual CPUs typically only map 4–6 physical cores of the first terminal host, resulting in limited available computing power.

[0183] When guest applications decode video in the virtual environment, they directly consume these already strained virtual CPU resources, easily causing playback stuttering, slow response, and other problems. Especially in scenarios with multiple emulator instances, each instance shares a limited CPU quota. If software decoding is still relied upon, it will further squeeze available computing power, exacerbate stuttering, and severely impact overall smoothness and user experience.

[0184] This application transmits the video stream from the Guest client of the mobile emulator to the Host client in real time via a communication channel, and the Host client directly calls the hardware decoder of the first terminal's graphics card for processing. In this way, the decoding task no longer relies on the limited virtual CPU on the Guest side, but fully utilizes the decoding resources of the idle and independent graphics processor on the first terminal host. This method effectively reduces the resource consumption of the CPU on the Guest side, significantly improves the video playback frame rate and the smoothness of the emulator application, thereby improving the user viewing experience and overall interactive performance. In other words, this application aims to solve the limitation that Android mobile emulators in the host environment cannot utilize the hardware decoding of the host graphics processor, proposing an efficient, low-overhead, and highly compatible technical path. By sending the video stream data from the Guest client of the mobile emulator to the hardware decoder on the Host side via a communication channel, the video playback of Android applications can directly use the first terminal's graphics card to perform hardware decoding, avoiding reliance on the software decoding of the virtual CPU on the Guest side, effectively reducing the resource consumption of the virtual CPU, and bringing a smoother playback experience and higher system resource utilization.

[0185] In practical applications, Figure 8 This is a schematic diagram of a mobile phone emulator simulating the running of a video application provided in an embodiment of this application. See also... Figure 8A mobile phone emulator is configured in the Windows runtime environment (i.e., host environment) of the first terminal (such as a laptop, desktop computer, etc.). The mobile phone emulator is used to simulate the running of video applications in the Android environment (i.e., virtual environment) of the mobile phone within the Windows runtime environment of the first terminal. In practical applications, the mobile phone emulator can have different product forms, that is, the mobile phone emulator can be manifested as an actual computer application, such as an application distribution client. When the user distributes the client in the Windows runtime environment of the first terminal, the first terminal responds to the startup operation of the distribution client, starts the distribution client, and displays the application distribution interface 110 of the distribution client. The application distribution interface 110 can display a startup entry for the Android environment of the mobile phone. The distribution client (equivalent to the mobile phone emulator, i.e., the Host end) responds to the startup operation triggered by the startup entry and simulates the running of the mobile phone virtual machine (Android runtime environment, i.e., virtual environment, i.e., the Guest end) in the Windows runtime environment. The mobile phone virtual machine supports users to download and start various video applications. In response to the startup operation of a certain video application, the mobile phone virtual machine can display the video playback interface 120 of the video application.

[0186] In practical applications, when a video application in a mobile phone emulator plays a video, the video processing method of this application embodiment can be used for video decoding. This involves transmitting the video stream from the Guest end (mobile phone virtual machine) of the mobile phone emulator to the Host end in real time via a communication channel, and then having the Host end directly call the hardware decoder of the first terminal graphics card for processing. Figure 9 This is a schematic diagram of the task manager provided in an embodiment of this application. See also: Figure 9 In the video decoding area 130, the graphics processing unit (GPU, i.e., hardware decoder) in the Task Manager executes the video decoding tasks of the video application. In this way, the video decoding task no longer relies on the limited virtual central processing unit on the guest side, effectively reducing the resource consumption on the guest side.

[0187] The video processing method of this application embodiment will now be described in detail using a mobile phone emulator.

[0188] The core of this application lies in modifying the decoder-related logic in the virtual environment so that the video playback data of each video application can be extracted on the Guest end and transmitted to the Host end. The Host end then calls the hardware decoder (i.e., graphics processor) in the Windows environment (i.e., the host environment) of the first terminal to perform hardware video decoding.

[0189] Figure 10 This is a schematic diagram of multi-terminal interaction provided in an embodiment of this application. See also... Figure 10The entire technical chain of the video processing method of this application can include three parts: the first part is the decoder initialization part, including S201 to S205; the second part is video decoding and playback, including S206 to S214; and the third part is the decoder destruction, including S215 to S218.

[0190] The first part will be explained, which includes S201 to S205.

[0191] S201, the video application starts, triggering the creation of a video decoder.

[0192] When the video application starts, it triggers the Guest side to call the decoder creation interface (such as the Mediacodec interface) to create a video decoder (i.e., a proxy decoder) for the video application.

[0193] S202, the Guest sends an initialization request to the Host.

[0194] Once the video decoder is created, an initialization operation for the video decoder can be triggered. Accordingly, the Guest can send an initialization request to the Host through a communication channel (i.e., a message pipeline, such as a pre-built Socket or Hyper-V channel).

[0195] S203, the host performs hardware capability testing.

[0196] In response to the initialization request, the host performs a hardware capability check on the currently available graphics processors.

[0197] S204 adaptively creates hardware decoder instances based on hardware capabilities and encoding standards.

[0198] S205, the Host sends an instance creation completion message to the Guest through the communication channel.

[0199] Here, we will first provide a detailed explanation of the creation of the video decoder in S201.

[0200] When a video application needs to play a video, it must first create and initialize a video decoder. Since the Guest (mobile virtual machine or container environment) cannot directly access the hardware decoding resources of the host machine (i.e., the first terminal of the Windows runtime environment), the decoder must be created and configured through the decoder creation interface (such as the MediaCodec interface) provided by the Android system AOSP on the Guest side. The decoder initialization process generally includes the following steps: 1. Obtain the decoder list (MediaCodec list) and select a suitable decoder; 2. Query all decoder components supported by the Android system through the codec directory (MediaCodecList), and filter out suitable decoder names based on video encoding format (such as H.264, H.265, VP9, ​​etc.), Multipurpose Internet Mail Extensions (MIME) type, and hardware acceleration requirements; 3. Create a video decoder instance, such as calling the name-based instance creation interface, such as MediaCodec.createByCodecName(String name) to create the corresponding decoder object on the Guest side, or calling the media format-based instance creation interface, such as MediaCodec.createDecoderByType(String type) to create the corresponding decoder object on the Guest side; 4. Configure video decoder parameters, construct the data stream format (MediaFormat), set key parameters such as video width and height, bitrate, frame rate, and color space, and call the configuration interface, such as configure(MediaFormat format, Surface surface, MediaCrypto crypto, int 5. Start the video decoder by calling the running interface, such as start(), to enter the running state. At this time, the input or output buffer queue inside the video decoder starts working.

[0201] The above process describes how the Android system on the Guest side creates a video decoder. In the mobile emulator scenario, the instance creation interface based on the media format creates a C2 decoder specifically for the mobile emulator, used to communicate with the Host side. Next, we will explain S202 to S204 in detail. After the Guest side calls the run interface (which can be automatically triggered when the video decoder is created, or triggered by the first video playback operation), it initializes the C2 decoder (i.e., the video decoder, also known as the aforementioned proxy decoder) and sends the initialization request (carrying an initialization data packet, i.e., decoding configuration parameters) to the Host side via a message pipeline. Upon receiving the initialization data packet, the host first enumerates the current video acceleration (Direct3D 11 Video Acceleration, D3D11VA) decoding capabilities of the first terminal. This involves the following steps: 1. Enumerating all display adapters (the core component being the graphics processor, i.e., the hardware decoder) on the first terminal using enumeration interfaces such as the EnumAdapters interface; 2. Creating corresponding hardware decoder instances (e.g., D3D11 devices) based on the enumerated display adapters using global functions (which create and return decoder instances, such as the D3D11CreateDevice function); 3. Obtaining the decoding capabilities of each display adapter through graphics card decoding capability query interfaces, such as using the GetVideoDecoderProfile function (for obtaining the hardware decoder's configuration file), CheckVideoDecoderFormat function (for obtaining the hardware decoder's format), and GetVideoDecoderConfig function (for obtaining the hardware decoder's specific decoder specification) of the ID3D11VideoDevice interface to obtain the graphics card's decoding capabilities; 4. Finally, enumerating the decoding capabilities of all current display adapters in the first terminal, including the resolution limit and supported encoding formats.

[0202] In practical applications, the initialization data packet also includes resolution and encoding format. The host compares the parameters in the initialization data packet with the enumerated decoding capabilities. If the resolution and encoding format in the initialization data packet are within the capabilities of a certain hardware decoder (display adapter), the created D3D11 device will be reserved for subsequent decoding. When the host completes the creation of the hardware decoder's decoding instance, it sends a success message back to the guest via a message pipe. Calling `start()` on the proxy decoder (i.e., the video decoder) will return a success message, completing the proxy decoder's initialization.

[0203] Below, regarding Figure 10The second part is explained, which includes S206 to S214.

[0204] S206, the video application receives a video playback command.

[0205] In response to a video playback operation, the video application calls the video decoding interface and submits the video stream to be decoded to the Guest for decoding. The video decoding interface then submits the video stream to the Guest's proxy decoder (i.e., the C2 decoder).

[0206] S207, the Guest-side proxy decoder sends the video stream to the Host-side via the message pipeline.

[0207] The proxy decoder (i.e., C2 decoder) does not directly perform video decoding. Instead, it is used to forward video stream-related operations and data to the host through a message pipeline.

[0208] S208: The host end adaptively calls the parsing interface corresponding to the encoding format to parse the video stream.

[0209] The host monitors the message pipeline in real time and performs corresponding operations based on the data transmitted through the message pipeline. Here, after receiving the video stream, it calls the parsing interface (such as the Parser interface) of the corresponding encoding format to parse the video stream.

[0210] S209, the host converts the video stream into data packets of a specified format and submits them to the decoding instance of the hardware decoder for decoding.

[0211] The host adaptively detects the hardware capabilities of the first terminal and calls the corresponding hardware decoding capability interface, such as DXVA2 or D3D11VA. It adaptively parses the video stream using different standard formats based on the video encoding format to convert the video stream into data packets recognizable by the hardware decoding capability interface (i.e., data packets of the specified format, i.e., the aforementioned target data format). Then, it submits the specified format data packets to the decoding instance for decoding.

[0212] In practical applications, the host can also be configured with a software decoder to provide a backup solution for hardware decoding failures.

[0213] S210, the Host sends the decoding result to the Guest's proxy decoder.

[0214] S211, the Guest's proxy decoder sends a video retrieval request to the Host.

[0215] S212, the host checks the decoding queue to see if there is a corresponding video frame that has been decoded.

[0216] S213, if applicable, the host will convert the color format of the decoded video frames.

[0217] The host side has the ability to convert color formats, ensuring that the host and guest sides use the same color format.

[0218] S214, The Host sends the format-converted and decoded video frames to the Guest's proxy decoder.

[0219] Here, the complete process from S206 to S214 is explained in detail.

[0220] After the video application starts and the proxy decoder (C2 decoder) is created, when the video application receives the video stream data, it can perform video decoding according to the following main actions, which can be divided into input and output parts.

[0221] Regarding the input process.

[0222] The proxy decoder obtains an available input buffer. For example, it calls the `dequeueInputBuffer` function to index an idle input buffer, gets the index of the input buffer, and then calls the `getInputBuffer` function to obtain the actual buffer (ByteBuffer). It then fills the input buffer with compressed video data, writing the received video stream data to it. Finally, it calls the `queueInputBuffer` function to submit the input buffer to the host's hardware decoder.

[0223] During the input process, the corresponding actions on the Host side are as follows: After the buffer commit function is called, the Guest-side proxy decoder sends the video stream data from the input buffer to the Host side via a message pipe for submission. Upon receiving the video stream data, the Host side first calls the corresponding encoding format parsing interface (such as the Parser interface) to analyze the video stream data, converts it to D3D11VA data packet format, and submits the video stream data to the decoding instance of the previously created hardware decoder via the decoder buffer commit function (such as SubmitDecoderBuffers) for video decoding. The decoded data is then cached in the decoding frame queue (i.e., the aforementioned decoding data queue). After the hardware decoder submission is complete, the Host side sends a successful submission message to the Guest side via a message pipe, at which point the buffer commit function (queueInputBuffer) returns a success message.

[0224] Regarding the output process.

[0225] The proxy decoder calls the output buffer acquisition function (such as dequeueOutputBuffer) to obtain the available output buffer; and obtains the byte buffer containing the decoded video data from the output buffer through the decoded data acquisition function (such as getOutputBuffer).

[0226] Here, the output process corresponds to the following actions on the Host side: When the proxy decoder calls the decoding data acquisition function (such as getOutputBuffer), the Guest-side proxy decoder sends a message requesting the acquisition of decoding data (i.e., a video acquisition request) to the Host side via a message pipeline. This message includes the color format required for decoding the video frame. After receiving the message to acquire decoding data, if the decoded video frame for the corresponding video exists in the decoded frame queue, the host side retrieves the earliest video frame that entered the decoded frame queue. The host side checks the color format of the retrieved video frame. If the color format of the requested video frame is inconsistent with the color format of the retrieved decoded video frame, a format conversion interface (such as libyuv) needs to be called to convert the color format. After the color format conversion is completed, the final decoded video frame is sent back to the Guest side via a message pipeline. At this time, the decoding data acquisition function (such as getOutputBuffer) will obtain the final decoded video data.

[0227] Next, regarding Figure 10 The third part of the document will be explained, which includes S215 to S218.

[0228] S215, closing the video application triggers the destruction of the video decoder.

[0229] S216, the video decoder on the Guest side sends a destruction request to the Host side.

[0230] The destruction request here is equivalent to the aforementioned close prompt message.

[0231] S217, the host side releases resources and destroys the decoding instance of the hardware decoder.

[0232] S218, the Host sends a destruction completion message to the Guest.

[0233] When a video application finishes playing a video, it needs to call a decoder destruction function (such as the `release` function) to destroy the video decoder. After calling the decoder destruction function, the Guest's C2 decoder (proxy decoder) will send a decoder destruction message (i.e., a destruction request) to the Host via a message pipeline. Upon receiving the decoder destruction message, the Host will release resources for the decoding instance of the hardware decoder managed by the Host, including releasing the D3D11 device and clearing the decoding buffer frame queue. After the release is complete, the Host will return a message indicating successful decoder destruction to the Guest, at which point the decoder destruction function will return a success message.

[0234] It should be noted that the message pipe mentioned above can be a Socket or Hyper-V channel to ensure low-latency, high-throughput data transmission and support concurrent operation of multiple applications. The data packets transmitted between the Guest-side proxy decoder and the Host-side via the message pipe can be of various types, such as initializing the decoder (InitContext), destroying the decoder (DestroyContext), decoding the image (DecodeImage), obtaining the image (GetImage), clearing the buffer (Flush), and resetting the decoder (Reset).

[0235] In this way, the proxy decoder transmits the video stream to the host in real time via a message pipeline and directly calls the host's graphics card hardware decoder for video decoding. In this way, the decoding task no longer relies on the guest's limited virtual CPU, but fully utilizes the host's idle and independent GPU decoding resources. This method effectively reduces the guest's CPU usage, significantly improves video playback frame rate and emulator application smoothness, thereby improving the user viewing experience and overall interactive performance.

[0236] The following description continues to illustrate the exemplary structure of the video processing apparatus 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the video processing device 455 in the memory 450 may include: The parameter receiving module 4551 is used to receive decoding configuration parameters for the video application sent by the proxy decoder running in the virtual environment; wherein, the video application in the virtual environment is simulated by the emulator in the host environment, and the proxy decoder is called by the video application in response to the video playback operation.

[0237] The instance creation module 4552 is used to determine the decoding instance of the target hardware decoder in the host environment when a target hardware decoder with adapted decoding configuration parameters is detected in the host environment.

[0238] The decoding module 4553 is used to decode the first video stream data originating from the video application by the target hardware decoder when it receives the first video stream data sent by the proxy decoder, in order to obtain the first decoded data.

[0239] The sending module 4554 is used to send the first decoded data to the proxy decoder so that the first decoded data can be sent to the video application for video playback via the proxy decoder.

[0240] In some embodiments, the parameter receiving module 4551 is further configured to parse the initialization request in response to receiving the initialization request sent by the proxy decoder, and obtain decoding configuration parameters; wherein, the proxy decoder is created based on the startup operation of the video application, and the initialization request is generated by the video application when it first responds to the video playback operation and calls the proxy decoder.

[0241] In some embodiments, the instance creation module 4552 is further configured to obtain multiple hardware decoders available in the host environment and create a decoding instance for each hardware decoder; determine the decoding capability information of the decoding instance of each hardware decoder, and detect whether each hardware decoder is compatible with the decoding configuration parameters based on the decoding capability information, and obtain the detection result; based on the detection result, select the target hardware decoder that is compatible with the decoding configuration parameters from among the multiple hardware decoders, and retain the decoding instance of the target hardware decoder.

[0242] In some embodiments, the instance creation module 4552 is further configured to obtain the load status of each hardware decoder that adapts to the decoding configuration parameters when there are multiple hardware decoders that adapt to the decoding configuration parameters; and to filter out the target hardware decoder whose load status is lower than the load threshold among the multiple hardware decoders that adapt to the decoding configuration parameters.

[0243] In some embodiments, before decoding the first video stream data through the decoding instance of the target hardware decoder, the instance creation module 4552 is further configured to send a first prompt message to the proxy decoder, so that the proxy decoder prompts the video application to send the first video stream data to the proxy decoder based on the first prompt message, and sends the first video stream data to the simulator through the proxy decoder; wherein, the first prompt message indicates that the decoding instance of the target hardware decoder has been successfully created.

[0244] In some embodiments, before decoding the first video stream data through the decoding instance of the target hardware decoder, the decoding module 4553 is further configured to perform format conversion on the first video stream data to obtain the first video stream data in a target data format; the target data format is a data format supported by the decoding instance of the target hardware decoder; and the first video stream data in the target data format is decoded through the decoding instance of the target hardware decoder.

[0245] In some embodiments, the sending module 4554 is further configured to receive a video acquisition request originating from a video application sent by the proxy decoder; parse the video acquisition request to obtain the target video identifier and target color format of the target video requested by the video application; when the video identifier corresponding to the first decoded data is the target video identifier, determine the color format of the first decoded data; when the color format of the first decoded data is different from the target color format, adjust the color format of the first decoded data to the target color format, and send the first decoded data of the target color format to the proxy decoder.

[0246] In some embodiments, the sending module 4554 is further configured to create a shader component in the host environment; bind the first decoded data as the input texture of the shader component; and call the decoding instance of the target hardware decoder to run the shader component, so as to perform color space conversion on the input texture through the shader component to generate the first decoded data conforming to the target color format.

[0247] In some embodiments, the sending module 4554 is further configured to send a second prompt message to the proxy decoder when the video identifier corresponding to the first decoded data is different from the target video identifier, so as to prompt the proxy decoder to send the second video stream data of the target video to the simulator through the second prompt message; wherein, the second prompt message indicates that there is no decoded data of the target video, and the second video stream data corresponds to the target video identifier; the decoding module 4553 is further configured to, when receiving the second video stream data, decode the second video stream data through the decoding instance of the target hardware decoder to obtain the second decoded data, and send the second decoded data to the proxy decoder through the sending module 4554, so as to send the second decoded data to the video application for video playback through the proxy decoder.

[0248] In some embodiments, the first decoded data is stored in a decoded data queue, and the host environment is configured with an input processing thread and an output processing thread running in parallel; the decoding module 4553 is also used to call the input processing thread to send the first video stream data to the decoding instance of the target hardware decoder, and to decode the first video stream data through the decoding instance of the target hardware decoder; and to call the output processing thread to store the first decoded data in the decoded data queue.

[0249] In some embodiments, after receiving the decoding configuration parameters for the video application sent by the proxy decoder running in the virtual environment, the instance creation module 4552 is further configured to create a decoding instance of the software decoder in the host environment when it is detected that there is no hardware decoder adapted to the decoding configuration parameters in the host environment, or when the total load of all hardware decoders in the host environment exceeds the load threshold; the decoding module 4553 is further configured to decode the first video stream data originating from the video application through the decoding instance of the software decoder when it receives the first video stream data sent by the proxy decoder, to obtain the third decoded data; the sending module 4554 is further configured to send the third decoded data to the proxy decoder so that the third decoded data can be sent to the video application for video playback through the proxy decoder.

[0250] In some embodiments, the decoding module 4553 is further configured to detect the running status of the decoding instance of the target hardware decoder in real time during the decoding of the first video stream data by the decoding instance of the target hardware decoder; when the running status indicates that the decoding instance of the target hardware decoder has encountered a decoding error, the decoding instance of the target hardware decoder is deregistered; the instance creation module 4552 is further configured to create a decoding instance of the software decoder in the host environment, so as to decode the first video stream data by the decoding instance of the software decoder.

[0251] In some embodiments, the sending module 4554 is further configured to call a pre-trained feature extraction model to extract features from the first decoded data to obtain decoded features; perform enhancement processing on the first decoded data based on the decoded features to obtain enhanced decoded data; and send the enhanced decoded data to the proxy decoder so that the enhanced decoded data can be sent to the video application for video playback through the proxy decoder.

[0252] In some embodiments, the video processing apparatus 455 further includes a deregistration module, configured to deregister the decoding instance of the target hardware decoder when receiving a shutdown prompt message sent by the proxy decoder for the video application; and to send a third prompt message to the proxy decoder when the deregistration of the decoding instance of the target hardware decoder is completed, so as to deregister the proxy decoder through the third prompt message; wherein the third prompt message indicates that the deregistration of the decoding instance of the target hardware decoder was successful.

[0253] In some embodiments, a shared memory shared with the virtual environment is configured in the host environment; the sending module 4554 is further configured to store the first decoded data in the shared memory and generate a corresponding data index identifier; and send the data index identifier to the proxy decoder so that the proxy decoder reads the first decoded data from the shared memory according to the data index identifier.

[0254] In some embodiments, the first decoded data is in a graphics texture format; the sending module 4554 is further configured to store the first decoded data in the graphics buffer of the target hardware decoder; map the graphics buffer to shared memory; or, transfer the first decoded data from the graphics buffer to the shared memory through a decoding instance of the target hardware decoder.

[0255] This application provides a computer program product, which includes a computer program or computer-executable instructions. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the video processing method provided in this application embodiment. For example, ... Figure 3 The video processing method is illustrated. The processor of the electronic device reads the computer program or computer-executable instructions from a computer-readable storage medium, and executes the computer program or computer-executable instructions, causing the electronic device to perform the video processing method described in the embodiments of this application.

[0256] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the video processing method provided in this application. For example, ... Figure 3 The video processing method shown.

[0257] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0258] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0259] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0260] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0261] In summary, through the embodiments of this application, when the emulator simulates and runs a video application in a virtual environment within the host environment, when the emulator receives decoding configuration parameters for the video application sent by the proxy decoder (called by the video application in response to video playback operations), if a target hardware decoder matching the decoding configuration parameters is detected in the host environment, the emulator determines the decoding instance of the target hardware decoder in the host environment. Upon receiving the first video stream data originating from the video application sent by the proxy decoder, the emulator decodes the first video stream data using the decoding instance of the target hardware decoder to obtain the first decoded data. Finally, the emulator returns the first decoded data to the proxy decoder so that the video application can play video based on the first decoded data. This method enables video decoding to be performed directly through the decoding instance of the hardware decoder in the host environment when the video application in the virtual environment plays video, reducing the resource consumption of the video application's central processing unit and thus improving video decoding efficiency, thereby improving the smoothness of video playback in the video application.

[0262] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A video processing method, characterized in that, An emulator applied in a host environment, the emulator being used to simulate and run a video application in a virtual environment within the host environment; the method includes: The system receives decoding configuration parameters for the video application sent by a proxy decoder running in the virtual environment; wherein the proxy decoder is invoked by the video application in response to a video playback operation. When a target hardware decoder that adapts to the decoding configuration parameters is detected in the host environment, a decoding instance of the target hardware decoder is determined in the host environment; When the first video stream data originating from the video application is received by the proxy decoder, the first video stream data is decoded by the decoding instance of the target hardware decoder to obtain the first decoded data; The first decoded data is sent to the proxy decoder so that the first decoded data can be sent to the video application for video playback via the proxy decoder.

2. The method according to claim 1, characterized in that, The step of receiving decoding configuration parameters for the video application sent by the proxy decoder running in the virtual environment includes: In response to receiving an initialization request sent by the proxy decoder, the initialization request is parsed to obtain the decoding configuration parameters; The proxy decoder is created based on the startup operation of the video application, and the initialization request is generated by the video application when it first responds to the video playback operation and calls the proxy decoder.

3. The method according to claim 1, characterized in that, When a target hardware decoder adapted to the decoding configuration parameters is detected in the host environment, determining the decoding instance of the target hardware decoder in the host environment includes: Obtain multiple hardware decoders available in the host environment, and create a decoding instance for each hardware decoder; Determine the decoding capability information of each hardware decoder's decoding instance, and detect whether each hardware decoder is compatible with the decoding configuration parameters based on the decoding capability information, and obtain the detection result; Based on the detection results, a target hardware decoder that matches the decoding configuration parameters is selected from the plurality of hardware decoders, and the decoding instance of the target hardware decoder is retained.

4. The method according to claim 3, characterized in that, The step of selecting a target hardware decoder that matches the decoding configuration parameters from the plurality of hardware decoders includes: When there are multiple hardware decoders that adapt to the decoding configuration parameters among the multiple hardware decoders, obtain the load status of each hardware decoder that adapts to the decoding configuration parameters; Among the multiple hardware decoders that adapt to the decoding configuration parameters, the target hardware decoder whose load state is lower than the load threshold is selected.

5. The method according to claim 1, characterized in that, Before decoding the first video stream data using the decoding instance of the target hardware decoder, the method further includes: A first prompt message is sent to the proxy decoder, so that the proxy decoder, based on the first prompt message, prompts the video application to send the first video stream data to the proxy decoder, and then sends the first video stream data to the emulator through the proxy decoder. The first prompt message indicates that the decoding instance of the target hardware decoder was successfully created.

6. The method according to claim 1, characterized in that, Before decoding the first video stream data using the decoding instance of the target hardware decoder, the method further includes: The first video stream data is converted to a new format to obtain a first video stream data in a target data format; the target data format is a data format supported by the decoding instance of the target hardware decoder. Decoding the first video stream data using a decoding instance of the target hardware decoder includes: The first video stream data of the target data format is decoded using the decoding instance of the target hardware decoder.

7. The method according to claim 1, characterized in that, Sending the first decoded data to the proxy decoder includes: Receives a video acquisition request originating from the video application sent by the proxy decoder; The video acquisition request is parsed to obtain the target video identifier and target color format of the target video requested by the video application; When the video identifier corresponding to the first decoded data is the target video identifier, the color format of the first decoded data is determined; When the color format of the first decoded data is different from the target color format, the color format of the first decoded data is adjusted to the target color format, and the first decoded data in the target color format is sent to the proxy decoder.

8. The method according to claim 7, characterized in that, Adjusting the color format of the first decoded data to the target color format includes: Create shader components in the host environment; The first decoded data is bound to the input texture of the shader component, and the decoding instance of the target hardware decoder is called to run the shader component, so as to perform color space conversion on the input texture through the shader component and generate the first decoded data that conforms to the target color format.

9. The method according to claim 7, characterized in that, The method further includes: When the video identifier corresponding to the first decoded data is different from the target video identifier, a second prompt message is sent to the proxy decoder, so as to prompt the proxy decoder to send the second video bitstream data of the target video to the simulator through the second prompt message; wherein, the second prompt message indicates that there is no decoded data of the target video, and the second video bitstream data corresponds to the target video identifier; When the second video stream data is received, the second video stream data is decoded by the decoding instance of the target hardware decoder to obtain the second decoded data, and the second decoded data is sent to the proxy decoder so that the second decoded data can be sent to the video application for video playback through the proxy decoder.

10. The method according to claim 1, characterized in that, The first decoded data is stored in a decoded data queue, and the host environment is configured with parallel-running input processing threads and output processing threads; Decoding the first video stream data using a decoding instance of the target hardware decoder includes: The input processing thread is invoked to send the first video stream data to the decoding instance of the target hardware decoder, and the first video stream data is decoded by the decoding instance of the target hardware decoder. After obtaining the first decoded data, the method further includes: The output processing thread is invoked to store the first decoded data into the decoded data queue.

11. The method according to claim 1, characterized in that, After receiving decoding configuration parameters for the video application sent by the proxy decoder running in the virtual environment, the method further includes: When it is detected that there is no hardware decoder adapted to the decoding configuration parameters in the host environment, or when the total load of all hardware decoders in the host environment exceeds the load threshold, a decoding instance of the software decoder is created in the host environment. When the first video stream data originating from the video application is received by the proxy decoder, the first video stream data is decoded by the decoding instance of the software decoder to obtain the third decoded data; The third decoded data is sent to the proxy decoder so that the proxy decoder can send the third decoded data to the video application for video playback.

12. The method according to claim 1, characterized in that, The method further includes: During the decoding process of the first video stream data by the decoding instance of the target hardware decoder, the running status of the decoding instance of the target hardware decoder is detected in real time. When the running status indicates that the decoding instance of the target hardware decoder has encountered a decoding error, the decoding instance of the target hardware decoder is deregistered. A decoding instance of a software decoder is created in the host environment to decode the first video stream data using the decoding instance of the software decoder.

13. The method according to claim 1, characterized in that, After obtaining the first decoded data, the method further includes: The pre-trained feature extraction model is invoked to extract features from the first decoded data to obtain the decoded features; The first decoded data is enhanced based on the decoding features to obtain enhanced decoded data. The enhanced decoding data is sent to the proxy decoder so that the enhanced decoding data can be sent to the video application for video playback via the proxy decoder.

14. The method according to claim 1, characterized in that, The method further includes: Upon receiving a shutdown prompt message from the proxy decoder for the video application, the decoding instance of the target hardware decoder is deregistered; When the decoding instance of the target hardware decoder is deregistered, a third prompt message is sent to the proxy decoder to deregister the proxy decoder through the third prompt message; The third prompt message indicates that the decoding instance of the target hardware decoder has been successfully deregistered.

15. The method according to claim 1, characterized in that, The host environment is configured with shared memory shared with the virtual environment; sending the first decoded data to the proxy decoder includes: The first decoded data is stored in the shared memory, and a corresponding data index identifier is generated; The data index identifier is sent to the proxy decoder so that the proxy decoder reads the first decoded data from the shared memory based on the data index identifier.

16. The method according to claim 15, characterized in that, The first decoded data is in a graphics texture format; storing the first decoded data in the shared memory includes: The first decoded data is stored in the graphics buffer of the target hardware decoder; Map the graphics buffer to the shared memory; or... The first decoded data is transferred from the graphics buffer to the shared memory via a decoding instance of the target hardware decoder.

17. A video processing apparatus, characterized in that, The device includes: The parameter receiving module is used to receive decoding configuration parameters for the video application sent by the proxy decoder running in the virtual environment; wherein the video application in the virtual environment is simulated by the emulator in the host environment, and the proxy decoder is invoked by the video application in response to video playback operations; The instance creation module is used to determine the decoding instance of the target hardware decoder in the host environment when a target hardware decoder that adapts to the decoding configuration parameters is detected in the host environment. The decoding module is used to decode the first video stream data originating from the video application by the target hardware decoder when it receives the first video stream data sent by the proxy decoder, in order to obtain the first decoded data. The sending module is used to send the first decoded data to the proxy decoder, so that the first decoded data can be sent to the video application for video playback through the proxy decoder.

18. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, configured to execute computer-executable instructions or computer programs stored in the memory, implements the video processing method according to any one of claims 1 to 16.

19. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the video processing method according to any one of claims 1 to 16 is implemented.

20. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the video processing method according to any one of claims 1 to 16 is implemented.

Images