Video coding method and system, computer device, storage medium, program product

Abstract

The embodiment of the present disclosure discloses a video coding method and system, computer equipment, storage medium and program product, wherein the video coding system comprises a controller, a host, a plurality of virtual machines, a host communication component, a plurality of virtual machine communication components corresponding to the plurality of virtual machines, and a plurality of video coding kernels; the controller communicates with the host through the host communication component; the controller communicates with each virtual machine through the corresponding virtual machine communication component; the controller is configured to, in response to a video coding request triggered by any of the plurality of virtual machines and the host, select a video coding kernel matched with the video coding request from the plurality of video coding kernels based on the resource state of the plurality of video coding kernels; and process a video coding task corresponding to the video coding request based on the matched video coding kernel.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of computer technology, and in particular to a video encoding and decoding method and system, computer equipment, storage medium, and program product. Background Technology

[0002] With the widespread adoption of ultra-high-definition and even 8K (K represents the number of pixels in horizontal resolution) resolution videos, the demand for video encoding and decoding has exploded, placing higher demands on the reliability and scalability of computer equipment's video encoding and decoding computing capabilities. Against this backdrop, virtualized video technology has emerged.

[0003] In related technologies, multiple virtual machines are configured on a host machine, each virtual machine is bound to a specified video codec kernel, and the video codec kernel is controlled by virtual machine management software. In this way, multiple video codec tasks can be processed through multiple virtual machines and their bound video codec kernels. However, this method suffers from the problem of uneven resource allocation. Summary of the Invention

[0004] In view of the above, the present disclosure provides at least one video encoding and decoding method and system, computer equipment, storage medium, and program product.

[0005] The technical solution of this disclosure embodiment is implemented as follows:

[0006] On one hand, this disclosure provides a video encoding and decoding system, which includes: a controller, a host, multiple virtual machines, a host communication component, multiple virtual machine communication components corresponding one-to-one with the multiple virtual machines, and multiple video encoding and decoding kernels;

[0007] The controller communicates with the host through a host communication component;

[0008] The controller communicates with each virtual machine through the corresponding virtual machine communication component;

[0009] The controller is used to respond to video encoding / decoding requests triggered by any of the multiple virtual machines and hosts. Based on the resource status of multiple video encoding / decoding kernels, it selects the video encoding / decoding kernel that matches the video encoding / decoding request from among the multiple video encoding / decoding kernels; and processes the video encoding / decoding task corresponding to the video encoding / decoding request based on the matched video encoding / decoding kernel.

[0010] On the other hand, embodiments of this disclosure provide a video encoding / decoding method applied to a video encoding / decoding system, the video encoding / decoding method comprising:

[0011] In response to video codec requests triggered by any of the multiple virtual machines and hosts, the system selects a video codec kernel that matches the video codec request based on the resource status of multiple video codec kernels.

[0012] The matching video codec kernel processes the video codec task corresponding to the video codec request.

[0013] In another aspect, embodiments of this disclosure provide a computer device, including a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0014] In another aspect, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0015] In another aspect, embodiments of this disclosure provide a computer program including computer-readable code, which, when executed in a computer device, causes a processor in the computer device to perform some or all of the steps in the above-described method.

[0016] In another aspect, embodiments of this disclosure provide a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method.

[0017] In this embodiment, there is no fixed binding method between the virtual machine and the video codec kernel; the controller manages all video codec kernels uniformly. When a video codec request is received from the host side (any one of multiple virtual machines and the host), the controller selects a matching video codec kernel based on its resource status to complete the video codec task corresponding to the request. This method of selecting a matching kernel based on the resource status of the video codec kernels ensures load balancing and prevents uneven resource allocation. Furthermore, through the corresponding communication components, the controller can not only receive video codec requests from the host side but also send the codec results back to the host side. Because the controller manages all video codec kernel resources uniformly, it can more flexibly and finely allocate the use of video codec kernels according to the needs of each virtual machine. When there is a need for high-resolution, high-frame-rate codec encoding and decoding, multiple video codec kernels can be flexibly scheduled to work in parallel to accelerate encoding and decoding.

[0018] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0020] Figure 1 A schematic diagram of the composition structure of a video encoding and decoding system provided in this embodiment of the disclosure. Figure 1 ;

[0021] Figure 2 A schematic diagram illustrating the implementation flow of a video encoding / decoding method provided in this embodiment of the disclosure;

[0022] Figure 3 A schematic diagram of the composition structure of a video encoding and decoding system provided in this embodiment of the disclosure. Figure 2 ;

[0023] Figure 4 A schematic diagram of the composition structure of a video encoding and decoding system provided in this embodiment of the disclosure. Figure 3 ;

[0024] Figure 5 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0026] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0027] The terms “first / second / third” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first / second / third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this disclosure.

[0029] The approach of binding virtual machines to specific video codec kernels in related technologies has the following problems: 1) Uneven resource allocation: In some cases, some video codec kernels may be idle, while others may experience performance bottlenecks due to heavy workloads. 2) Lack of global coordination: Each video codec kernel uses resources according to its own pace and task requirements, making it difficult to achieve efficient resource scheduling among multiple video codec kernels to support global virtualization. 3) High management complexity: As the number of video codec kernels increases, configuring, monitoring, and maintaining each kernel individually becomes very complex, which may lead to higher maintenance costs and longer maintenance times.

[0030] Therefore, embodiments of this disclosure provide a video encoding / decoding system, such as... Figure 1 As shown, the video encoding / decoding system 10 includes: a controller 11, a host 12, multiple virtual machines 13, a host communication component 14, multiple virtual machine communication components 15 corresponding one-to-one with the multiple virtual machines, and multiple video encoding / decoding kernels 16; the controller 11 communicates with the host 12 through the host communication component 14; the controller 11 communicates with each of the virtual machines 13 through the corresponding virtual machine communication component 15; the controller 11 is used to respond to a video encoding / decoding request triggered by any of the multiple virtual machines 13 and the host 12, and select a video encoding / decoding kernel from the multiple video encoding / decoding kernels 16 that matches the video encoding / decoding request based on the resource status of the multiple video encoding / decoding kernels 16; and process the video encoding / decoding task corresponding to the video encoding / decoding request based on the matched video encoding / decoding kernel.

[0031] The video encoding / decoding system 10 is used to handle video encoding / decoding tasks. Video encoding / decoding tasks are key processes that involve processing video data through operations such as encoding (compression) and decoding (decompression) to achieve efficient storage, transmission, and real-time processing while maintaining visual quality.

[0032] The video encoding and decoding system 10 can be deployed in a computer device, which can be a server, server cluster, laptop, tablet, desktop computer, smart TV, set-top box, mobile device, vehicle device, wearable device, or other device with data processing capabilities.

[0033] The controller 11 manages the video codec cores 16. In some embodiments, the controller 11 comprehensively understands the resource status of all video codec cores 16 to achieve unified allocation and optimization of the video codec cores 16; it also performs unified task scheduling of the video codec cores 16 to promote collaborative work among them and achieve load balancing. Resource status includes information such as resource usage and task load. Resource usage includes whether the video codec core is idle and its resource utilization rate; task load includes computational power consumption and performance metrics. For example, the controller 11 can be a microcontroller unit (MCU).

[0034] like Figure 1 As shown, the controller 11 can act as a central coordinator to better schedule multiple video codec kernels 16 to process video codec tasks, improving the system performance of the video codec kernels 16 and enhancing the system's processing capabilities. Furthermore, the controller 11 enables resource isolation of the video codec kernels 16. Simultaneously, the controller 11 acts as a security control point, implementing a unified security policy for each virtual instance of the video codec kernel 16; thus, system security is enhanced.

[0035] Host 12 refers to the physical computer on which virtualization software (such as VMware, VirtualBox, or Hyper-V) is installed. It runs directly on the hardware and provides resources and environment to support the operation of virtual machines through the virtualization software. The operating system (Host OS) in Host 12 is the original operating system installed on the physical computer, running directly on the hardware and managing the hardware resources of the entire computer device, including the graphics processing unit (GPU), memory, storage devices, network interfaces, etc.

[0036] Virtual Machine 13 is an independent computer environment simulated by virtualization software. It runs on the host machine and uses virtual hardware resources (such as virtual GPU and virtual memory) provided by the virtualization software to run its own operating system and applications. The operating system (Guest OS) in Virtual Machine 13 runs through the virtual hardware environment simulated by the virtualization software. The virtual machine operating system can be any operating system that supports virtualization, such as Windows, Linux, macOS, etc.

[0037] The host communication component 14 is used to realize communication between the host 12 and the controller 11. The host communication component 14 can be a register component for communication between the host 12 and the controller 11. For example, the host communication component 14 can be a video host interaction interface (VHI) between the video codec core and the host.

[0038] Virtual machine communication component 15 is used to implement communication between virtual machine 13 and controller 11. Virtual machine communication component 15 can be a register component for communication between virtual machine 13 and controller 11. For example, virtual machine communication component 15 can be a video codec core and virtual machine interaction interface (VVI).

[0039] The video codec kernel 16 is used to perform the operations involved in video codec tasks. It supports a variety of codec standards (such as H.264, Advanced Audio Coding (AAC), etc.) to provide efficient processing capabilities.

[0040] Video encoding / decoding requests can be triggered by either party on the host side, where "any party" refers to the host or any virtual machine. Specifically, host 12 can issue host video encoding / decoding requests through host communication component 14, and virtual machine 13 can issue virtual machine video encoding / decoding requests through virtual machine communication component 15.

[0041] It should be noted that, Figure 1 This example uses four virtual machines; in real-world scenarios, more virtual machines and corresponding pathways can be configured based on business needs. Figure 1 As can be seen, each OS (including the host OS and virtual machine OS) has an independent communication component to communicate with the controller 11. Each communication component provides bidirectional transmission and interrupt enabling functions between each OS and the MCU system. When virtualization is not enabled, the host OS can communicate with the controller 11 through the host communication component 14; when virtualization is enabled, each virtual machine OS can communicate with the controller 11 through its corresponding virtual machine communication component.

[0042] In this embodiment, there is no fixed binding method between the virtual machine and the video codec kernel; the controller manages all video codec kernels uniformly. When a video codec request is received from the host side (any one of multiple virtual machines and the host), the controller selects a matching video codec kernel based on its resource status to complete the video codec task corresponding to the request. This method of selecting a matching kernel based on the resource status of the video codec kernels ensures load balancing and prevents uneven resource allocation. Furthermore, through the corresponding communication components, the controller can not only receive video codec requests from the host side but also send the codec results back to the host side. Because the controller manages all video codec kernel resources uniformly, it can more flexibly and finely allocate the use of video codec kernels according to the needs of each virtual machine. When there is a need for high-resolution, high-frame-rate codec encoding and decoding, multiple video codec kernels can be flexibly scheduled to work in parallel to accelerate encoding and decoding.

[0043] In some embodiments, the controller 11 runs a response service; the response service is configured to allocate a first memory space for a video encoding / decoding request triggered by the host 12, and return the address data of the first memory space to the host through the host communication component 14; the response service is also configured to allocate a second memory space for a video encoding / decoding request triggered by the virtual machine 13, and return the address data of the second memory space to the virtual machine through the virtual machine communication component 15.

[0044] The response service is used to handle requests from the host side. Specifically, the response service responds to video encoding / decoding requests issued by the host side, allocates memory space for the video encoding / decoding requests, and also sends encoding / decoding completion messages through the corresponding communication component. For example, the response service can be a HostService.

[0045] The first memory space refers to the memory space allocated for video encoding / decoding requests issued by the host. The second memory space refers to the memory space allocated for video encoding / decoding requests issued by the virtual machine. Memory space can be represented as TaskBuff. There is a one-to-one correspondence between memory space and video encoding / decoding tasks.

[0046] In some implementations, when the host OS on the host side has video encoding / decoding requirements, the host OS can send a request to the controller through the host communication component 14. The response service (HostService) in the controller 11 allocates a TaskBuff (a segment of memory used to store specific configuration information issued by the host side, including raw image data, temporary data generated when the video encoding / decoding kernel is working, and data after the final encoding / decoding is completed) on the video memory (e.g., Double Data Rate Synchronous Dynamic Random Access Memory (DDR)). Then, the address of the TaskBuff is returned to the host OS through the host communication component 14.

[0047] In some implementations, when the virtual machine 13 on the host side has video encoding and decoding requirements, the virtual machine 13 can send a request to the controller 11 through the virtual machine communication component 15. The response service (HostService) in the controller 11 allocates a TaskBuff on the video memory, and then returns the address of the TaskBuff to the virtual machine through the virtual machine communication component.

[0048] In some embodiments, the video encoding / decoding system 10 further includes: a virtual machine manager 17; the virtual machine manager 17 is configured to store relevant data of video encoding / decoding requests triggered by the virtual machine into the second memory space through a communication interface, so as to implement the distribution of video encoding / decoding tasks corresponding to the virtual machine; the host 12 is configured to store relevant data of video encoding / decoding requests triggered by the host into the first memory space through the communication interface, so as to implement the distribution of video encoding / decoding tasks corresponding to the host; the relevant data includes: configuration information and raw image data.

[0049] The virtual machine manager 17 is used to create, manage, and monitor the operation of virtual machines. The host sends video encoding / decoding requests to the controller 11 and stores the relevant data of the video encoding / decoding requests in the corresponding memory space; this can be understood as completing the distribution of the video encoding / decoding task.

[0050] The transmission methods for video encoding / decoding requests and their associated data differ. Specifically, the video encoding / decoding request from host 12 is sent through host communication component 14, while the associated data can be directly stored in the first memory space. The video encoding / decoding request from virtual machine 13 is sent through virtual machine communication component 15, while the associated data can be stored in the second storage space via virtual machine manager 17. After the host sends the configuration information and raw image data for this encoding / decoding task to the corresponding memory space, it can notify controller 11 that the video encoding / decoding task has been completed via the corresponding communication component.

[0051] In some embodiments, the video encoding / decoding system 10 includes a task pool; the task pool is used to store unprocessed video encoding / decoding tasks; the response service is further used to store the memory space corresponding to the video encoding / decoding task into the task pool after the video encoding / decoding task has been issued.

[0052] The memory space corresponding to the video encoding and decoding task is stored in the task pool; that is, the video encoding and decoding task is stored in the task pool.

[0053] In some embodiments, the controller 11 runs a video codec service; the video codec service is used to obtain the resource status of the plurality of video codec kernels 16, the resource status including: whether the plurality of video codec kernels 16 are idle, computing power consumption, and performance indicators; the video codec service is also used to select a currently pending video codec task from a task pool based on the task load and task priority that the idle video codec kernel can bear when there is an idle video codec kernel among the plurality of video codec kernels 16; control the idle video codec kernel to process the pending video codec task; the task pool is used to store unprocessed video codec tasks.

[0054] The video codec service is used to control the video codec kernel 16 to implement specific video codec tasks. For example, the video codec service can be VPUService.

[0055] When there are many video encoding and decoding tasks, unprocessed video encoding and decoding services can be temporarily stored in the task pool. The video encoding and decoding service in the controller monitors in real time whether the video encoding and decoding kernel is idle. If an idle video encoding and decoding kernel is found, the video encoding and decoding task to be processed is selected from the task pool according to the task priority (the video encoding and decoding service matches the task load that the idle video encoding and decoding kernel can handle), and the idle video encoding and decoding kernel processes the video encoding and decoding task to be processed.

[0056] In some embodiments, the first memory space is further used to store temporary data generated during the processing of the video encoding and decoding task of the host 12 and the processed final image data; the response service is further used to transmit an encoding and decoding end message to the host 12 through the host communication component 14 after the video encoding and decoding task of the host 12 has been processed, so that the host 12 can retrieve the final image data from the first memory space; the response service is further used to reclaim the first memory space through the host communication component 14 after the host 12 has retrieved the final image data.

[0057] After the video encoding and decoding task of host 12 is completed, the host communication component 14 transmits the encoding and decoding end message to host 12 so that host 12 can retrieve the data from the first memory space and notify controller 11 after retrieving the data so that the response service in controller 11 can reclaim the first memory space.

[0058] Correspondingly, the second memory space is also used to store temporary data generated during the processing of the video encoding and decoding task of the virtual machine 13 and the final image data after processing; the response service is also used to transmit an encoding and decoding end message to the virtual machine 13 through the virtual machine communication component 15 after the video encoding and decoding task of the virtual machine 13 is completed, so that the virtual machine 13 can retrieve the final image data from the second memory space; after the virtual machine 13 retrieves the final image data, it reclaims the second memory space through the virtual machine communication component 15.

[0059] In some embodiments, the controller 11 is further configured to integrate the resources of the plurality of video codec kernels 16; the resources include computing resources, memory resources, and bandwidth resources; the controller 11 is further configured to uniformly update the firmware and parameter configuration of the plurality of video codec kernels 16 when the firmware and parameter configuration of the plurality of video codec kernels 16 need to be updated.

[0060] By introducing an independent controller 11, the computing power (computing resources), memory resources, and bandwidth resources of each video codec kernel 16 are abstracted to form a virtual resource set. The controller 11 dynamically allocates video codec kernel resources to different virtual machines according to different users or application scenarios to maximize resource utilization. Furthermore, the controller 11 can perform firmware updates and parameter configurations on some or all video codec kernels. Compared to managing each video codec kernel individually, this effectively reduces management costs and complexity, and improves system maintainability.

[0061] This disclosure provides a video encoding / decoding method, which is applied to a video encoding / decoding system 10. For example... Figure 2As shown, the method includes the following steps 201 to 202:

[0062] Step 201: Respond to video encoding / decoding requests triggered by any of the multiple virtual machines and hosts, and select a video encoding / decoding kernel that matches the video encoding / decoding request from the multiple video encoding / decoding kernels based on the resource status of the multiple video encoding / decoding kernels.

[0063] In some implementations, step 201 can be specifically implemented as follows: based on the resource status of the plurality of video codec kernels, determine the idle video codec kernels among the plurality of video codec kernels; determine the target number of video codec kernels required when the video codec request is processed; if the number of idle video codec kernels meets the target number, determine the target number of video codec kernels from the idle video codec kernels based on a preset selection strategy to obtain the matching video codec kernels.

[0064] The target number refers to the number of video codec cores required to process a video codec request. A number of idle video codec cores satisfying the target number means that the number of idle video codec cores is greater than or equal to the target number. A preset selection strategy is used to select video codec cores that match the video codec request when there are a large number of idle video codec cores. Preset selection strategies may include, but are not limited to: random selection strategies, optimal selection strategies based on performance priorities, balanced selection strategies based on idle time duration, and balanced selection strategies based on a combination of performance and idle time.

[0065] In some implementations, if the number of idle video codec cores does not meet the target number, the video codec task corresponding to the video codec request is stored in a task pool until the number of idle video codec cores meets the target number. Then, the video codec request is processed based on the target number of idle video codec cores. If, while the video codec task is stored in the task pool, there are other video codec requests that need to be processed, and the number of video codec cores required for these other requests is less than the target number, and the number of idle video codec cores is sufficient to process the remaining requests, then the remaining requests can be processed first. This maximizes the utilization of video codec core resources.

[0066] Step 202: Process the video encoding / decoding task corresponding to the video encoding / decoding request based on the matched video encoding / decoding kernel.

[0067] In this embodiment, there is no fixed binding method between the virtual machine and the video codec kernel; the controller manages all video codec kernels uniformly. When a video codec request is received from the host side (any one of multiple virtual machines and the host), the controller selects a matching video codec kernel based on its resource status to complete the video codec task corresponding to the request. This method of selecting a matching kernel based on the resource status of the video codec kernels ensures load balancing and prevents uneven resource allocation. Furthermore, through the corresponding communication components, the controller can not only receive video codec requests from the host side but also send the codec results back to the host side. Because the controller manages all video codec kernel resources uniformly, it can more flexibly and finely allocate the use of video codec kernels according to the needs of each virtual machine. When there is a need for high-resolution, high-frame-rate codec encoding and decoding, multiple video codec kernels can be flexibly scheduled to work in parallel to accelerate encoding and decoding.

[0068] The following describes the application of the video encoding and decoding method provided in this disclosure in real-world scenarios.

[0069] This disclosure provides an effective and flexible configuration management method for multi-channel video codec kernels, which can solve problems such as uneven resource allocation, lack of global coordination, complex management, and difficulty in global virtualization.

[0070] like Figure 3 As shown, four video codec kernels are listed, but more can be configured according to business needs in actual design. This embodiment of the disclosure achieves flexible configuration and management of the video codec kernels by adding a set of MCUs and a register component (the aforementioned communication component) for communication between the MCU and the host. On the graphics card side, each OS (including the host OS and virtual machine OS) has an independent register component for communication with the MCU. This register provides bidirectional information transmission and interrupt enabling functions between each OS and the MCU. When virtualization is not enabled, the host OS communicates with the MCU through VHI (the aforementioned host communication component). When virtualization is enabled, each virtual machine communicates with the MCU through VVI (the aforementioned virtual machine communication component).

[0071] By adding an independent MCU and corresponding register components, flexible configuration and management of the video codec kernels are achieved. The MCU performs unified task scheduling for all video codec kernels. When it receives a video codec request from the host side, it selects an idle video codec kernel to complete the corresponding video codec request and feeds back the encoding and decoding results to the corresponding OS on the host side via VHI or VVI, thus achieving load balancing of video codec kernel encoding resources. When virtualization is enabled, because the MCU manages all video codec kernel resources in a unified manner, the use of video codec kernels can be more flexibly and finely divided according to the needs of each virtual machine. There is no fixed binding between virtual machines and video codec kernels. When there is a need for high-resolution, high-frame-rate encoding and decoding, multiple video codec kernels can be flexibly scheduled to work in parallel, thereby accelerating encoding and decoding.

[0072] like Figure 4 As shown, two services run on the MCU: the HostService (the aforementioned response service) handles requests on the host side, and the VPUService (the aforementioned video encoding / decoding service) controls and schedules the video encoding / decoding kernel to complete specific video encoding / decoding tasks.

[0073] When there is a video encoding / decoding requirement from the host OS or virtual machine on the host side, the host system sends a request to the MCU via VHI or VVI. The HostService in the MCU allocates a TaskBuff (a segment of memory used to store the specific configuration information issued by the host side, the original image data, as well as the temporary data generated when the video encoding / decoding kernel is working and the final image data after the encoding / decoding is completed) on the video memory (DDR), and then returns the address of the TaskBuff to the host side via VHI or VVI.

[0074] The host sends the configuration information and raw image data for this encoding / decoding task to the TaskBuff, and then notifies the MCU that the task has been sent via VHI or VVI. The HostService in the MCU puts the TaskBuff into the TaskPool (a task pool used to store encoding / decoding tasks from the host side that are yet to be processed). When the VPUService finds an idle video encoding / decoding kernel, it selects a task from the TaskPool according to the task priority and assigns it to the corresponding idle video encoding / decoding kernel to complete the specific encoding / decoding task. After the video encoding / decoding kernel completes the task, it notifies the MCU. The HostService in the MCU sends an encoding / decoding completion message to the host side via VHI or VVI. The host application retrieves the data from the TaskBuff and then notifies the MCU. The MCU HostService then reclaims the TaskBuff.

[0075] The embodiments disclosed herein can achieve at least the following technical effects:

[0076] 1) Unified resource management and optimization; the MCU can fully understand the working status of all video codec kernels, including resource usage, task load, etc., thereby realizing unified resource allocation and optimization, and achieving load balancing.

[0077] 2) Enhanced collaboration capabilities; it can promote collaborative work between video codec cores. The MCU can act as a central coordinator, enabling multiple video codec cores to better share information and collaboratively handle complex tasks.

[0078] 3) Simplified system monitoring and maintenance: Unified management via MCU makes monitoring and maintenance of the entire system much easier. The MCU can collect status information from each video codec core, such as power consumption and performance metrics, and can perform firmware updates and parameter configurations uniformly with newly added MCUs. This is much simpler than managing each video codec core individually, effectively reducing system management costs and complexity, and improving system maintainability.

[0079] 4) Facilitates virtualization; by introducing an independent MCU, the computing power, memory resources and bandwidth resources of each video codec kernel are abstracted to form a virtual resource set, and the video codec kernel resources are dynamically allocated to different virtual instances according to different users or application scenarios.

[0080] 5) Enhanced isolation and security: Through MCU management, the isolation of video codec kernel resources can be better achieved in the virtualization environment. At the same time, the MCU can act as a security control point to implement a unified security policy for each video codec kernel virtual instance.

[0081] 6) No modifications are needed to the already verified video codec kernel, reducing project risk.

[0082] 7) It increases the flexibility of the software, enabling flexible resource allocation.

[0083] The innovations of this disclosure include at least the following: by adding a set of MCUs and designing a set of independent communication components for each system (including host OS and virtual machine OS), a flexible configuration management method for multi-channel video codec kernels is achieved.

[0084] It should be noted that, in the embodiments of this disclosure, if the above-described video encoding and decoding method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this disclosure are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0085] This disclosure provides a computer device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0086] This disclosure provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium may be transient or non-transient.

[0087] This disclosure provides a computer program including computer-readable code, wherein when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0088] This disclosure provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0089] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referenced interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0090] It should be noted that, Figure 5 This is a schematic diagram of a hardware entity of a computer device in an embodiment of this disclosure, such as... Figure 5 As shown, the hardware entity of the computer device 500 includes: a processor 501, a communication interface 502, and a memory 503, wherein:

[0091] Processor 501 typically controls the overall operation of computer device 500.

[0092] Communication interface 502 enables computer devices to communicate with other terminals or servers via a network.

[0093] The memory 503 is configured to store instructions and applications executable by the processor 501, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 501 and various modules in the computer device 500. It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 501, the communication interface 502, and the memory 503 can be performed via bus 504.

[0094] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above embodiments of this disclosure are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0095] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0096] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0097] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0098] In addition, each functional unit in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0099] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0100] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0101] The above description is merely an embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A video encoding and decoding system, characterized in that, The video codec system includes: a host and multiple virtual machines on the host side, a controller on the graphics card side, a host communication component, multiple virtual machine communication components corresponding one-to-one with the multiple virtual machines, and multiple video codec kernels; the controller is directly connected to the multiple video codec kernels to achieve at least one of the following functions: obtaining the resource status of the multiple video codec kernels, isolating the resources of the multiple video codec kernels, uniformly implementing the security policies of the virtual instances of the multiple video codec kernels, firmware updates and parameter configurations of some or all video codec kernels, unified allocation and optimization of the multiple video codec kernels, and unified task scheduling of the multiple video codec kernels; The controller communicates with the host via the host communication component; The controller communicates with each virtual machine through a corresponding virtual machine communication component; The controller is configured to respond to video encoding / decoding requests triggered by any of the plurality of virtual machines and the host, select a video encoding / decoding kernel from the plurality of video encoding / decoding kernels that matches the video encoding / decoding request based on the resource status of the plurality of video encoding / decoding kernels, and process the video encoding / decoding task corresponding to the video encoding / decoding request based on the matched video encoding / decoding kernel.

2. The video encoding and decoding system according to claim 1, characterized in that, The controller runs a responsive service. The response service is used to allocate a first memory space for the video encoding / decoding request triggered by the host, and return the address data of the first memory space to the host through the host communication component; The response service is also used to allocate a second memory space for the video encoding / decoding request triggered by the virtual machine, and return the address data of the second memory space to the virtual machine through the virtual machine communication component.

3. The video encoding and decoding system according to claim 2, characterized in that, The video encoding / decoding system also includes: a virtual machine manager; The virtual machine manager is used to store the relevant data of the video encoding and decoding request triggered by the virtual machine to the second memory space through the communication interface, so as to realize the distribution of the video encoding and decoding task corresponding to the virtual machine; The host is used to store the relevant data of the video encoding / decoding request triggered by the host into the first memory space through the communication interface, so as to realize the distribution of the video encoding / decoding task corresponding to the host; the relevant data includes: configuration information and raw image data.

4. The video encoding and decoding system according to claim 3, characterized in that, The video encoding / decoding system includes a task pool; the task pool is used to store unprocessed video encoding / decoding tasks. The response service is also used to store the memory space corresponding to the video encoding and decoding task into the task pool after the video encoding and decoding task has been issued.

5. The video encoding and decoding system according to any one of claims 1 to 4, characterized in that, The controller runs a video encoding / decoding service. The video codec service is used to obtain the resource status of the plurality of video codec kernels, the resource status including: whether the plurality of video codec kernels are idle, computing power consumption, and performance indicators; The video codec service is further configured to, when there are idle video codec kernels among the plurality of video codec kernels, select a currently pending video codec task from the task pool based on the task load and task priority that the idle video codec kernel can bear; control the idle video codec kernel to process the pending video codec task; the task pool is used to store unprocessed video codec tasks.

6. The video encoding and decoding system according to claim 3, characterized in that, The first memory space is also used to store temporary data generated during the processing of the host's video encoding and decoding tasks and the final image data after processing; The response service is also used to transmit a decoding completion message to the host through the host communication component after the host has finished processing the video encoding and decoding task, so that the host can retrieve the final image data from the first memory space. The response service is also used to reclaim the first memory space through the host communication component after the host has taken away the final image data.

7. The video encoding and decoding system according to any one of claims 1 to 4, or 6, characterized in that, The controller is also used to integrate the resources of the multiple video codec kernels; the resources include computing resources, memory resources, and bandwidth resources. The controller is also configured to uniformly update the firmware and parameter configuration of the multiple video codec kernels when the firmware and parameter configuration of the multiple video codec kernels need to be updated.

8. A video encoding / decoding method, characterized in that, An application is made in a video encoding / decoding system, which includes: a host and multiple virtual machines on the host side, a controller on the graphics card side, a host communication component, multiple virtual machine communication components corresponding one-to-one with the multiple virtual machines, and multiple video encoding / decoding kernels; the controller is directly connected to the multiple video encoding / decoding kernels to achieve at least one of the following functions: acquiring the resource status of the multiple video encoding / decoding kernels, isolating the resources of the multiple video encoding / decoding kernels, uniformly implementing the security policies of the virtual instances of the multiple video encoding / decoding kernels, firmware updates and parameter configurations of some or all video encoding / decoding kernels, unified allocation and optimization of the multiple video encoding / decoding kernels, and unified task scheduling of the multiple video encoding / decoding kernels; the video encoding / decoding method includes: In response to a video codec request triggered by any of the plurality of virtual machines and the host, a video codec kernel matching the video codec request is selected from the plurality of video codec kernels based on the resource status of the plurality of video codec kernels; Based on the matched video codec kernel, the video codec task corresponding to the video codec request is processed.

9. The video encoding and decoding method according to claim 8, characterized in that, The step of selecting a video codec kernel that matches the video codec request from among the multiple video codec kernels based on their resource status includes: Based on the resource status of the plurality of video codec kernels, determine the video codec kernels that are idle among the plurality of video codec kernels; Determine the target number of video codec kernels required for processing the video codec request; If the number of idle video codec kernels meets the target number, a target number of video codec kernels are determined from the idle video codec kernels based on a preset selection strategy to obtain the matching video codec kernels.

10. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 8 to 9.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 8 to 9.

12. A computer program product comprising a non-transitory computer-readable storage medium storing a computer program, wherein when the computer program is read and executed by a computer, it implements the steps of the method of any one of claims 8 to 9.

Images