Virtualized video encoding and decoding system and method, electronic device and storage medium

By adopting two-level hardware flow and hardware configuration isolation technology in the virtualized video codec system, the problem of increasing response time when switching video standards is solved is solved, processing performance and security are improved, and more efficient video virtualization is achieved to expand scalability.

WO2024235288A1PCT designated stage expired Publication Date: 2024-11-21MOORE THREADS TECH CO LTD

Patent Information

Application Number
PCT/CN2024/093651
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-16
Filing Date
2024-05-16
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

The existing virtualized video codec system increases response time when switching different video standards, and the computing performance decreases, and the software configuration method cannot completely isolate the virtual machine, which poses security and reliability risks.

Method used

Use two-level hardware flow method to perform video encoding and decoding tasks, realize hardware configuration isolation through virtual machine registers, decouple configuration process and codec calculation process, and use the storage management module to allocate exclusive storage space for each video encoding and codec task to reduce virtual machines The burden of the manager.

Benefits of technology

It improves the scalability and processing performance of video virtualization, reduces the complexity of the virtual machine manager, and enhances the security of data access and system reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024093651_21112024_PF_FP_ABST
    Figure CN2024093651_21112024_PF_FP_ABST
Patent Text Reader

Abstract

The present disclosure relates to a virtualized video encoding and decoding system and method, an electronic device, and a storage medium. The system comprises: a virtual machine manager, a virtual machine, a virtual machine register corresponding to the virtual machine, and a video encoding and decoding kernel. The virtual machine manager is configured to configure, to the virtual machine register, configuration information of a video encoding and decoding task corresponding to the virtual machine, wherein the video encoding and decoding task corresponds to the j-th video standard, and j is a positive integer greater than or equal to 1; the video encoding and decoding kernel is configured to read the configuration information from the virtual machine register, and execute the video encoding and decoding task according to the j-th video standard and on the basis of the configuration information. The embodiments of the present disclosure can flexibly execute a plurality of video encoding and decoding tasks corresponding to different video standards in a virtual machine.
Need to check novelty before this filing date? Find Prior Art

Description

A virtualized video encoding and decoding system and method, electronic device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure is based on and claims the priority of Chinese patent application with application number 2023105526563 and application date May 16, 2023. The entire contents of the Chinese patent application are hereby introduced into this application in their entirety. Technical Field

[0003] The present disclosure relates to the field of computer technology, and in particular to a virtualized video encoding and decoding system and method, an electronic device, and a storage medium. Background Art

[0004] Virtualization is a resource management technology that abstracts and divides computer hardware and software system resources, allowing multiple virtual machines to reuse a set of, but not limited to, a limited set of system resources. With the explosive growth in the demand for video encoding and decoding in network servers in recent years, the video encoding and decoding computing power in graphics processing units (GPUs) chips has become increasingly important. In order to enhance the security, reliability, and scalability of system encoding and decoding tasks, virtualized video technology has emerged. Virtualized video technology is reflected in the simultaneous processing of multiple independent video encoding and decoding tasks on the same set of physical resources. For example, in the scenario of virtual desktops, a large number of users are operating and processing videos of different video standards at the same time. Therefore, there is an urgent need for a virtualized video encoding and decoding system with efficient video encoding and decoding processing capabilities.

[0005] Summary of the Invention

[0006] The present disclosure proposes a technical solution of a virtualized video encoding and decoding system and method, an electronic device, and a storage medium.

[0007] According to one aspect of the present disclosure, a virtualized video codec system is provided, comprising: a virtual machine manager, a virtual machine, a virtual machine register corresponding to the virtual machine, and a video codec kernel; the virtual machine manager is configured to configure configuration information of a video codec task corresponding to the virtual machine to the virtual machine register, wherein the video codec task corresponds to the jth video standard, and j is a positive integer greater than or equal to 1; the video codec kernel is configured to read the configuration information from the virtual machine register, and based on the configuration information, execute the video codec task in accordance with the jth video standard.

[0008] In a possible implementation, the system further includes: a virtual machine interface corresponding to the virtual machine; and the virtual machine manager is further configured to configure the configuration information to the virtual machine register based on the virtual machine interface.

[0009] In one possible implementation, the configuration information includes: the encoding and decoding parameters corresponding to the video encoding and decoding task, and the storage space size required to execute the video encoding and decoding task; the virtual machine register includes: parameter configuration and memory management modules for M video standards, where M is a positive integer greater than or equal to 2, and the M video standards include the j-th video standard; the parameter configuration and memory management module for the j-th video standard is configured to determine the encoding and decoding parameters corresponding to the video encoding and decoding task, and the storage space size required to execute the video encoding and decoding task.

[0010] In one possible implementation, the system further includes: a storage management module and a storage unit; the storage management module is configured to allocate a corresponding target storage space for the video encoding and decoding task in the storage unit based on the storage space size required to perform the video encoding and decoding task.

[0011] In a possible implementation, the video codec core performs data access based on a target storage space corresponding to the video codec task during the execution of the video codec task.

[0012] In a possible implementation, different video encoding and decoding tasks have different task numbers.

[0013] In one possible implementation, the virtual machine register further includes: an interrupt module; the interrupt module is configured to send an interrupt signal to the virtual machine manager based on the virtual machine interface after executing the i-th video encoding and decoding task, wherein the interrupt signal is used to indicate that the i-th video encoding and decoding task is completed, wherein i is a positive integer greater than or equal to 1.

[0014] In one possible implementation, the virtual machine manager is further configured to control the video codec core to execute the i+1th video codec task after receiving the interrupt signal, wherein the configuration information of the i+1th video codec task has been configured into the virtual machine register.

[0015] In a possible implementation, when the system includes multiple virtual machines, different virtual machines correspond to different virtual machine interfaces, and different virtual machines correspond to different virtual machine registers.

[0016] In a possible implementation, different virtual machine video encoding and decoding tasks corresponding to one virtual machine have data access permissions; and virtual video encoding and decoding tasks corresponding to different virtual machines do not have data access permissions.

[0017] According to one aspect of the present disclosure, a virtualized video codec method is provided, comprising: a virtual machine manager configuring configuration information of a video codec task corresponding to a virtual machine to a virtual machine register corresponding to the virtual machine, wherein the video codec task corresponds to a j-th video standard, and j is a positive integer greater than or equal to 1; a video codec kernel reading the configuration information from the virtual machine register, and executing the video codec task according to the j-th video standard based on the configuration information.

[0018] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0019] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above method is implemented.

[0020] In an embodiment of the present disclosure, a virtualized video codec system includes: a virtual machine manager, a virtual machine, a virtual machine register corresponding to the virtual machine, and a video codec kernel; the virtual machine manager configures configuration information of a video codec task corresponding to the virtual machine into the virtual machine register, wherein the video codec task corresponds to the jth video standard, and j is a positive integer greater than or equal to 1; the video codec kernel is configured to read the configuration information from the virtual machine register, and based on the configuration information, execute the video codec task according to the jth video standard. Utilizing the virtualized video codec system of the embodiment of the present disclosure, it is possible to flexibly execute multiple video codec tasks corresponding to different video standards in one virtual machine, thereby effectively improving the scalability of video virtualization, so that the capabilities of video virtualization can be utilized in more scenarios.

[0021] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, rather than limiting the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0023] FIG1 is a schematic diagram showing a virtualized video encoding and decoding system in the related art;

[0024] FIG2 is a schematic diagram showing a virtualized video codec in the related art;

[0025] FIG3 shows a schematic diagram of a virtualized video encoding and decoding system according to an embodiment of the present disclosure;

[0026] FIG4 shows a schematic diagram of virtualized video encoding and decoding according to an embodiment of the present disclosure;

[0027] FIG5 shows a schematic diagram of a virtualized video encoding and decoding system according to an embodiment of the present disclosure;

[0028] FIG6 shows a flowchart of a virtualized video encoding and decoding method according to an embodiment of the present disclosure;

[0029] FIG7 shows a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0031] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0032] The term "and / or" herein simply describes an association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent the existence of three situations: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0033] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0034] In related art, virtualized video encoding and decoding tasks for different video standards are primarily implemented through software configuration. Figure 1 shows a schematic diagram of a virtualized video encoding and decoding system in related art. As shown in Figure 1, the process for implementing virtualized video encoding and decoding tasks for different video standards through software configuration is as follows.

[0035] Step 1: Initiate a video codec task for the first video standard. The virtual machine manager queries the operating status of all virtual machines in the current virtualized video codec system, including but not limited to virtual machine information, whether the video codec task has been completed, and status information such as occupied storage space. The virtualized video codec system may include N virtual machines, where N is a positive integer greater than or equal to 2. As shown in Figure 1, the software layer includes the virtual machine manager 100, virtual machines 301, 302, 303, and so on, and the hardware layer includes the video codec kernel 200. The virtual machine manager 100 queries the operating status of virtual machines 301, 302, 303, and so on, and so on.

[0036] Step 2: To avoid overlapping storage space for different video encoding and decoding tasks, the virtual machine manager allocates dedicated storage space for the current virtual machine for the current video encoding and decoding task based on the storage space occupied by all other virtual machines obtained through the query, and adds the virtual machine information and the corresponding storage space to the management queue of the virtual machine manager for subsequent query use.

[0037] Step 3: The virtual machine manager issues a series of configuration commands to the video codec core, including querying, configuring parameters, and allocating storage space. This process sends information such as the current virtual machine's virtual machine information and required storage space to the video codec core via configuration registers. As shown in Figure 1, parameter configuration and memory management for different video standards are implemented in software at the software layer.

[0038] Step 4: The virtual machine manager issues control commands to the video codec kernel, including but not limited to obtaining the original video data or encoded bitstream corresponding to the video codec task, encoding and decoding the data or bitstream, and other commands.

[0039] Step 5: The video codec kernel performs the corresponding video codec task operations according to the configuration and commands of the virtual machine manager. During this process, the data access space and storage space are limited to the dedicated storage space configured in advance by the virtual machine manager for the current virtual machine.

[0040] Step 6: After the video codec kernel completes the video codec task of the first video standard, it interacts with the virtual machine manager by sending an interrupt signal to inform the virtual machine manager that the video codec task of the first video standard has been completed.

[0041] Step 7: After receiving the interrupt signal from the video codec kernel, the virtual machine manager may start the video codec task of the next video standard.

[0042] When implementing virtualized video codec tasks for different video standards through software configuration, switching between different video standards increases the response time and timing uncertainty of the virtualized video codec system, impacting the computing performance of the virtualized video codec system. The isolated processing of different video standards further complicates the precise scheduling of the virtual machine manager within the virtualized video codec system. Furthermore, software configuration cannot completely isolate different virtual machines within the virtualized video codec system, posing a high risk to security and reliability. First, to ensure the exclusivity of virtual machine process storage and avoid overlap between different virtual machine processes, the virtual machine manager must perform extensive maintenance, querying, and scheduling tasks. This workload increases with the number of supported virtual machine processes. Second, because there is only one interrupt signal between the video codec kernel and the host, the virtual machine manager must poll the interrupted virtual machine process and perform maintenance, record the reasons for the virtual machine process's completion, and query it at appropriate times. This places an excessive burden on the virtual machine manager and increases management costs, while also slowing down its response to other interrupt signals within the virtualized video codec system. Third: The configuration process of the video codec kernel by the virtual machine manager is dependent on the calculation process of the video codec kernel. The video codec kernel can only perform the codec calculation process after the virtual machine manager completes the codec parameter configuration process.

[0043] FIG2 is a schematic diagram of virtualized video encoding and decoding in the related art. As shown in FIG2 , for the video encoding and decoding task 1 corresponding to the virtual machine 301, the virtual machine manager performs the configuration process of the video encoding and decoding task 1 corresponding to the virtual machine 301 according to the video standard 1 (as shown in FIG2 : virtual machine 301 / video standard 1 / configuration process), and then the video encoding and decoding kernel can perform the encoding and decoding calculation of the video encoding and decoding task 1 corresponding to the virtual machine 301 according to the video standard 1 (as shown in FIG2 : virtual machine 301 / video standard 1 / encoding and decoding calculation); for the video encoding and decoding task 2 corresponding to the virtual machine 302, the virtual machine manager performs the configuration process of the video encoding and decoding task 2 corresponding to the virtual machine 302 according to the video standard 2 (as shown in FIG2 : virtual machine 302 / video standard 2 / configuration process), and then the video encoding and decoding kernel can perform the encoding and decoding calculation of the video encoding and decoding task 2 corresponding to the virtual machine 302 according to the video standard 2 (as shown in FIG2 : virtual machine 302 / video standard 2 / encoding and decoding calculation ); for video codec task 3 corresponding to virtual machine 303, the virtual machine manager performs the configuration process for video codec task 3 corresponding to virtual machine 303 according to video standard 3 (as shown in Figure 2: virtual machine 303 / video standard 3 / configuration process), and then the video codec core can perform the codec calculations for video codec task 3 corresponding to virtual machine 303 according to video standard 3 (as shown in Figure 2: virtual machine 303 / video standard 3 / codec calculation); until, for video codec task n corresponding to virtual machine 30N, the virtual machine manager performs the configuration process for video codec task n corresponding to virtual machine 30N according to video standard n (as shown in Figure 2: virtual machine 30N / video standard n / configuration process), and then the video codec core can perform the codec calculations for video codec task n corresponding to virtual machine 30N according to video standard n (as shown in Figure 2: virtual machine 30N / video standard n / codec calculation). This serial operating mode, which relies on software configuration and hardware encoding in sequence, affects the performance of the video codec core. In addition, when switching between different video standards, the virtual machine manager needs to reconfigure the codec parameters and calculate the storage space size according to the different video standards, which will also increase the response time of the virtualized video codec system.

[0044] In order to solve the problem of implementing virtualized video codecs for different video standards through software configuration, the embodiments of the present disclosure provide a virtualized video codec system with efficient video codec processing capabilities. The virtualized video codec system provided by the present disclosure is described in detail below.

[0045] Figure 3 shows a schematic diagram of a virtualized video codec system according to an embodiment of the present disclosure. As shown in Figure 3, the virtualized video codec system includes: a virtual machine manager 100, a virtual machine 300, a virtual machine register 500 corresponding to the virtual machine, and a video codec core 200. The virtual machine manager 100 is configured to allocate configuration information of a video codec task corresponding to the virtual machine 300 to the virtual machine register 500, where the video codec task corresponds to the jth video standard, where j is a positive integer greater than or equal to 1. The video codec core 200 is configured to read the configuration information from the virtual machine register and, based on the configuration information, execute the video codec task according to the jth video standard.

[0046] The video codec tasks corresponding to a virtual machine may refer to the video codec tasks that the virtual machine needs to perform to meet the needs of the virtual machine user. The specific number of video codec tasks corresponding to a virtual machine and the specific form of the video standard corresponding to each video codec task can be flexibly changed according to actual needs and are not specifically limited in this disclosure.

[0047] By utilizing the virtualized video codec system of the disclosed embodiment, multiple video codec tasks corresponding to different video standards can be flexibly executed in a virtual machine, thereby effectively improving the scalability of video virtualization and enabling the video virtualization capabilities to be utilized in more scenarios.

[0048] In a possible implementation, different video encoding and decoding tasks have different task numbers.

[0049] When a virtual machine corresponds to n video codec tasks, different task numbers are used to distinguish different video codec tasks, where n is a positive integer greater than or equal to 1. For example, a virtual machine corresponds to three video codec tasks: video codec task 1, video codec task 2, and video codec task 3.

[0050] The task numbers of different video encoding and decoding tasks can be in any form according to actual needs, and this disclosure does not make any specific limitations on this.

[0051] A video codec task is the basic unit of video codec operations and consists of two steps: configuration and codec calculation. In the disclosed embodiments, a two-stage hardware pipeline is used to execute these two steps. The first stage, hardware, performs the video codec task configuration; the second stage, the video codec core, performs the codec calculation. The following details each of these steps.

[0052] In a possible implementation, the virtualized video codec system further includes: a virtual machine interface corresponding to the virtual machine; and a virtual machine manager further configured to configure configuration information to a virtual machine register based on the virtual machine interface.

[0053] In a virtualized video codec system, each virtual machine corresponds to a virtual machine interface, which in turn corresponds to a virtual machine register. Based on the virtual machine interface, the virtual machine manager assigns the configuration information for the video codec task corresponding to the virtual machine to the virtual machine register corresponding to the virtual machine interface, thereby achieving configuration isolation based on hardware configuration.

[0054] As shown in FIG. 3 , the virtual machine manager 100 configures the configuration information of the video encoding and decoding task corresponding to the virtual machine 300 to the corresponding virtual machine register 500 based on the virtual machine interface 400 corresponding to the virtual machine 300 .

[0055] In one possible implementation, the configuration information includes: codec parameters corresponding to a video codec task, and the size of the storage space required to perform the video codec task; the virtual machine register includes: a parameter configuration and memory management module for M video standards, where M is a positive integer greater than or equal to 2, and the M video standards include the jth video standard; the parameter configuration and memory management module for the jth video standard is configured to determine the codec parameters corresponding to the video codec task, and the size of the storage space required to perform the video codec task.

[0056] The virtual machine registers corresponding to a virtual machine in a virtualized video codec system include parameter configurations for M video standards and a memory management module, enabling the virtual machine to perform video codec tasks for any of the M video standards based on the hardware's virtual machine registers. This improves the scalability of video virtualization, enabling it to be utilized in more scenarios. The M video standards can be common video standards in the video codec field, and this disclosure does not limit the specific value of M or the specific form of the M video standards.

[0057] As shown in Figure 3, the virtual machine 300 includes parameter configuration and management modules for M video standards based on the virtual machine register 500 connected to the virtual machine interface 400: a parameter configuration and management module 501 for the first video standard, a parameter configuration and management module 502 for the second video standard, a parameter configuration and management module 503 for the third video standard, and up to a parameter configuration and management module 50M for the Mth video standard.

[0058] As shown in Figure 3, virtual machine 300 may include virtual machine 301, virtual machine 302, virtual machine 303, and up to virtual machine 30N; virtual machine interface 400 may include virtual machine interface 401, virtual machine interface 402, virtual machine interface 403, and up to virtual machine interface 40N; virtual machine register 500 may include virtual machine register 510, virtual machine register 520, virtual machine register 530, and up to virtual machine register 5N0, where the specific value of N can be determined according to actual conditions and is not specifically limited in this disclosure.

[0059] After starting the configuration process of the i-th (i is a positive integer greater than or equal to 1) video codec task corresponding to a virtual machine, if the i-th video codec task corresponds to the j-th video standard, at this time, based on the virtual machine interface, call the parameter configuration and management module of the j-th video standard in the corresponding virtual machine register to determine the configuration information corresponding to the i-th video codec task: the codec parameters corresponding to the i-th video codec task, and the storage space size required to execute the i-th video codec task.

[0060] Furthermore, the configuration information corresponding to the i-th video codec task is stored in the virtual machine register, so that after the codec calculation process of the i-th video codec task is started, the video codec core reads the configuration information from the virtual machine register, and based on the configuration information, performs the codec calculation of the i-th video codec task according to the j-th video standard.

[0061] During the encoding and decoding calculation process of the i-th video encoding and decoding task, the configuration process of the i+1-th video encoding and decoding task corresponding to the virtual machine can be started synchronously. If the i+1-th video encoding and decoding task corresponds to the j+1-th video standard, at this time, based on the virtual machine interface, the parameter configuration and management module of the j+1-th video standard in the corresponding virtual machine register is called to determine the configuration information corresponding to the i+1-th video encoding and decoding task: the encoding and decoding parameters corresponding to the i+1-th video encoding and decoding task, and the storage space size required to execute the i+1-th video encoding and decoding task.

[0062] Furthermore, the configuration information corresponding to the i+1th video codec task is stored in the virtual machine register, so that after executing the codec calculation of the i-th video codec task, since the configuration of the i+1th video codec task has been completed, the codec calculation process of the i+1th video codec task can be directly started, and the video codec core reads the configuration information from the virtual machine register, and based on the configuration information, executes the codec calculation of the i+1th video codec task according to the j+1th video standard.

[0063] Compared with the software configuration method, the serial working module of the configuration process and encoding and decoding calculation process of the next video encoding and decoding task can be started only after the configuration process and encoding and decoding calculation process of one video encoding and decoding task are completed. The embodiment of the present disclosure is based on the hardware virtual machine register, which can decouple the configuration process and encoding and decoding calculation process of the video encoding and decoding task, so that when the encoding and decoding calculation of one video encoding and decoding task is executed, the configuration process of other video encoding and decoding tasks can be processed in parallel, thereby effectively improving the processing performance of the video encoding and decoding core.

[0064] For example, a virtual machine corresponds to n video codec tasks. First, configure the first video codec task. After the configuration is complete, execute the codec calculations for the first video codec task. At this time, the configuration process of other video codec tasks can be processed in parallel. For example, the second video codec task. After the codec calculations for the first video codec task are completed, the codec calculations for the second video codec task can be executed directly because the configuration of the second video codec task has already been completed.

[0065] Figure 4 shows a schematic diagram of virtualized video encoding and decoding according to an embodiment of the present disclosure. As shown in Figure 4, virtual machine 301 corresponds to n video encoding and decoding tasks: virtual machine 301 / video encoding and decoding task 1, virtual machine 301 / video encoding and decoding task 2, virtual machine 301 / video encoding and decoding task 3, and so on. The configuration process of the n video encoding and decoding tasks is decoupled from the codec calculation process. The n video encoding and decoding tasks are configured sequentially: configuring video encoding and decoding task 1, configuring video encoding and decoding task 2, configuring video encoding and decoding task 3, and so on. Video encoding and decoding calculations are performed sequentially on the n video encoding and decoding tasks: codec calculations for virtual machine 301 / video standard 1 / video encoding and decoding task 1, codec calculations for virtual machine 301 / video standard 2 / video encoding and decoding task 2, codec calculations for virtual machine 301 / video standard 3 / video encoding and decoding task 3, and so on.

[0066] In one possible implementation, the virtualized video encoding and decoding system further includes: a storage management module and a storage unit; the storage management module is configured to allocate corresponding target storage space for the video encoding and decoding task in the storage unit based on the storage space size required to perform the video encoding and decoding task.

[0067] After starting the encoding and decoding calculation process of the i-th video encoding and decoding task, the video encoding and decoding core reads the configuration information of the i-th video encoding and decoding task from the virtual machine register: the encoding and decoding parameters corresponding to the i-th video encoding and decoding task, and the storage space size required to execute the i-th video encoding and decoding task. The video encoding and decoding core sends the storage space size required for the i-th video encoding and decoding task to the storage management module, so that the storage management module allocates the corresponding target storage space for the i-th video encoding and decoding task in the storage unit.

[0068] In a possible implementation, target storage spaces for video encoding and decoding tasks corresponding to different virtual machines are different.

[0069] Compared with the software configuration method in which the virtual machine manager consumes time and computing power to maintain, traverse, and query the occupancy status of the storage space in the current storage unit to avoid storage space overlap, the embodiment of the present application is based on the storage management module and uses hardware logic to allocate exclusive storage space for different video encoding and decoding tasks under a virtual machine, effectively realizing the storage space isolation between video encoding and decoding tasks corresponding to different virtual machines, effectively ensuring the security of data access, and in addition, freeing the query scheduling work of the virtual machine manager and reducing the complexity requirements of the virtual machine manager.

[0070] In one example, when a virtual machine corresponds to n video encoding and decoding tasks, the storage unit includes exclusive storage space allocated to the n video encoding and decoding tasks: the target storage space corresponding to the first video encoding and decoding task, the target storage space corresponding to the second video encoding and decoding task, and so on, until the target storage space corresponding to the nth video encoding and decoding task.

[0071] In a possible implementation, the video codec core performs data access based on the target storage space corresponding to the video codec task during the execution of the video codec task.

[0072] The video codec core performs the codec calculation of the i-th video codec task based on the configuration information of the i-th video codec task read from the virtual machine register. During this process, data is accessed based on the target storage space corresponding to the i-th video codec task in the storage unit.

[0073] In one possible implementation, the virtual machine register further includes: an interrupt module; the interrupt module is configured to send an interrupt signal to the virtual machine manager based on the virtual machine interface after executing the i-th video encoding and decoding task, wherein the interrupt signal is used to indicate that the i-th video encoding and decoding task is completed, wherein i is a positive integer greater than or equal to 1.

[0074] After completing the i-th video encoding and decoding task, the video codec core sends an interrupt request to the interrupt module in the virtual machine register, so that the interrupt module generates an interrupt signal based on the interrupt request, and then sends an interrupt signal to the virtual machine register based on the virtual machine interface to notify the virtual machine manager that the i-th video encoding and decoding task is completed.

[0075] In a virtual machine in a virtualized video codec system, different video codec tasks reuse the same interrupt module in the corresponding virtual machine register, thereby effectively reducing the waiting time of the system interrupt process. As shown in Figure 3, the virtual machine register 500 includes an interrupt module 600.

[0076] In a possible implementation, the virtual machine manager is further configured to control the video codec core to execute the (i+1)th video codec task after receiving the interrupt signal, wherein the configuration information of the (i+1)th video codec task has been configured in the virtual machine register.

[0077] After the virtual machine manager determines that the i-th video codec task has been completed based on the received interrupt signal, due to the aforementioned configuration isolation method, the configuration information of the i+1-th video codec task has been configured into the virtual machine register. Therefore, the virtual machine manager can start the codec calculation process of the i+1-th video codec task, that is, control the video codec core to perform the codec calculation of the i+1-th video codec task.

[0078] In a possible implementation, when a virtualized video codec system includes multiple virtual machines, different virtual machines correspond to different virtual machine interfaces, and different virtual machines correspond to different virtual machine registers.

[0079] In a possible implementation, different virtual machine video encoding and decoding tasks corresponding to one virtual machine have data access permissions; and virtual video encoding and decoding tasks corresponding to different virtual machines do not have data access permissions.

[0080] The number of multiple virtual machines included in the virtualized video encoding and decoding system can be adjusted according to actual needs of the system, and this disclosure does not make any specific restrictions on this.

[0081] By utilizing the hardware's virtual machine registers, configuration isolation between different virtual machines can be achieved, as well as decoupling between the configuration process and video encoding and decoding processing, thereby improving the processing performance of the video encoding and decoding core. The exclusive storage space corresponding to the video encoding and decoding tasks of each virtual machine is different, that is, the virtual video encoding and decoding tasks corresponding to different virtual machines do not have data access rights, so that the video encoding and decoding tasks of each virtual machine do not affect each other, effectively improving data security.

[0082] Figure 5 shows a schematic diagram of a virtualized video codec system according to an embodiment of the present disclosure. As shown in Figure 5, the virtualized video codec system includes: a virtual machine manager 100, virtual machines 301, 302, 303, and so on, up to virtual machine 30N. The specific value of N can be determined based on actual conditions and is not specifically limited by this disclosure. Different virtual machines correspond to different virtual machine interfaces to connect to different virtual machine registers. As shown in Figure 5, virtual machine 301 connects to virtual machine register 510 based on virtual machine interface 401, virtual machine 302 connects to virtual machine register 520 based on virtual machine interface 402, virtual machine 303 connects to virtual machine register 530 based on virtual machine interface 403, and up to virtual machine 30N connect to virtual machine register 5N0 based on virtual machine interface 40N. As shown in Figure 5, each virtual machine register includes parameter configuration and management modules for M video standards: parameter configuration and management module 501 for the first video standard, parameter configuration and management module 502 for the second video standard, parameter configuration and management module 503 for the third video standard, and up to parameter configuration and management module 50M for the Mth video standard, as well as an interrupt module 600. As shown in FIG5 , the virtualized video codec system further includes a video codec core 200 .

[0083] Each of the N virtual machines in the virtualized video codec system can handle multiple video codec tasks. The virtual machine manager uses hardware-based virtual machine interfaces and registers to isolate configuration and interrupts between different virtual machines. Compared to software configuration, the virtual machine manager eliminates the complex query and scheduling tasks, reducing software configuration time. Each virtual machine only needs to maintain its own configuration information and interrupt signals, and different virtual machines are unaware of each other, achieving more complete virtualization.

[0084] For example, the virtual machine manager configures the configuration information of the video encoding and decoding task corresponding to the virtual machine 301 to the virtual machine register 510 based on the virtual machine interface 401, and returns the interrupt signal corresponding to the virtual machine 301 in the virtual machine register 510 to the virtual machine manager based on the virtual machine interface 401; the virtual machine manager configures the configuration information of the video encoding and decoding task corresponding to the virtual machine 302 to the virtual machine register 520 based on the virtual machine interface 402, and returns the interrupt signal corresponding to the virtual machine 302 in the virtual machine register 520 to the virtual machine manager based on the virtual machine interface 402; and so on, which will not be repeated here.

[0085] The process of each virtual machine executing video encoding and decoding tasks of multiple different video standards can refer to the relevant processes of the embodiments shown in Figures 3 and 4 above, and will not be described in detail here.

[0086] FIG6 shows a flowchart of a virtualized video encoding and decoding method according to an embodiment of the present disclosure. The method can be applied to the above-mentioned virtualized video encoding and decoding system. As shown in FIG6 , the method includes:

[0087] In step S61 , the virtual machine manager configures configuration information of the video encoding and decoding task corresponding to the virtual machine into a virtual machine register corresponding to the virtual machine, wherein the video encoding and decoding task corresponds to the j-th video standard, and j is a positive integer greater than or equal to 1.

[0088] In step S62 , the video codec core reads configuration information from the virtual machine register, and performs the video codec task according to the j-th video standard based on the configuration information.

[0089] The configuration and encoding and decoding process of the video encoding and decoding task can refer to the relevant processes of the embodiments shown in Figures 3 to 5 above, and will not be described in detail here.

[0090] It is understood that the above-mentioned various method embodiments mentioned in this disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to space limitations, this disclosure will not go into details. It is understood by those skilled in the art that in the above-mentioned methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0091] In addition, the present disclosure also provides electronic devices, computer-readable storage media, and programs, all of which can be used to implement any virtualized video encoding and decoding method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method section and will not be repeated here.

[0092] This method has a specific technical connection with the internal structure of the computer system, and can solve the technical problem of how to improve the hardware computing efficiency or execution effect (including reducing the amount of data storage, reducing the amount of data transmission, increasing the hardware processing speed, etc.), thereby obtaining the technical effect of improving the internal performance of the computer system in accordance with the laws of nature.

[0093] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0094] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0095] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0096] An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0097] The electronic device may be provided as a terminal, a server, or other forms of devices.

[0098] FIG7 shows a block diagram of an electronic device according to an embodiment of the present disclosure. Referring to FIG7 , the electronic device 1900 can be provided as a server or a terminal device. Referring to FIG7 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions that can be executed by the processing component 1922, such as an application. The application stored in the memory 1932 can include one or more modules, each of which corresponds to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0099] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (Mac OS X TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open-source Unix-like operating system (FreeBSD TM ) or similar.

[0100] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0101] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0102] Computer-readable storage media can be a tangible device that can hold and store the instructions used by the instruction execution device. Computer-readable storage media can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. Computer-readable storage media used herein is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (for example, light pulses by fiber optic cables), or electrical signals transmitted by wires.

[0103] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0104] The computer program instructions for performing the operation of the present disclosure can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions can be executed entirely on a user's computer, partially on a user's computer, executed as an independent software package, partially on a user's computer and partially on a remote computer, or executed entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, by utilizing the state information of computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0105] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0106] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0107] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0108] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0109] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0110] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0111] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0112] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0113] While various embodiments of the present disclosure have been described above, the above descriptions are intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A virtualized video encoding and decoding system, the system comprising: A virtual machine manager, a virtual machine, a virtual machine register corresponding to the virtual machine, and a video codec kernel; The virtual machine manager is configured to configure configuration information of the video encoding and decoding task corresponding to the virtual machine to the virtual machine register, wherein the video encoding and decoding task corresponds to the j-th video standard, and j is a positive integer greater than or equal to 1; The video codec core is configured to read the configuration information from the virtual machine register, and based on the configuration information, perform the video codec task according to the j-th video standard.

2. The system according to claim 1, further comprising: A virtual machine interface corresponding to the virtual machine; The virtual machine manager is further configured to configure the configuration information to the virtual machine register based on the virtual machine interface.

3. The system according to claim 1 or 2, wherein the configuration information comprises: The encoding and decoding parameters corresponding to the video encoding and decoding task, and the storage space size required to perform the video encoding and decoding task; The virtual machine register includes: a parameter configuration and memory management module for M video standards, wherein M is a positive integer greater than or equal to 2, and the M video standards include the jth video standard; The parameter configuration and memory management module of the jth video standard is configured to determine the encoding and decoding parameters corresponding to the video encoding and decoding task, and the storage space size required to execute the video encoding and decoding task.

4. The system according to claim 3, further comprising: Storage management module, storage unit; The storage management module is configured to allocate a corresponding target storage space for the video encoding and decoding task in the storage unit based on the storage space size required to execute the video encoding and decoding task.

5. The system according to claim 4, wherein the video codec core performs data access based on a target storage space corresponding to the video codec task during the execution of the video codec task.

6. The system according to any one of claims 1 to 5, wherein different video encoding and decoding tasks have different task numbers.

7. The system according to claim 6, wherein the virtual machine register further comprises: Interrupt module; The interrupt module is configured to send an interrupt signal to the virtual machine manager based on the virtual machine interface after executing the i-th video encoding and decoding task, wherein the interrupt signal is used to indicate that the i-th video encoding and decoding task is completed, wherein i is a positive integer greater than or equal to 1.

8. The system according to claim 7, wherein the virtual machine manager is further configured to control the video codec core to execute the (i+1)th video codec task after receiving the interrupt signal, wherein: The configuration information of the (i+1)th video encoding and decoding task has been configured into the virtual machine register.

9. The system according to claim 2, wherein when the system includes multiple virtual machines, different virtual machines correspond to different virtual machine interfaces, and different virtual machines correspond to different virtual machine registers.

10. The system according to claim 9, wherein different virtual machine video encoding and decoding tasks corresponding to a virtual machine have data access rights; Virtual video encoding and decoding tasks corresponding to different virtual machines do not have data access permissions.

11. A virtualized video encoding and decoding method, comprising: The virtual machine manager configures configuration information of the video encoding and decoding task corresponding to the virtual machine to the virtual machine register corresponding to the virtual machine, wherein the video encoding and decoding task corresponds to the j-th video standard, and j is a positive integer greater than or equal to 1; The video codec kernel reads the configuration information from the virtual machine register, and based on the configuration information, executes the video codec task according to the j-th video standard.

12. An electronic device comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method of claim 11.

13. A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the method of claim 11 when executed by a processor.

Citation Information

Patent Citations

  • Apparatus and method for hybrid layer of address mapping for virtualized input / output implementation

    CN109690505A

  • Virtualized video coding and decoding system, electronic equipment and storage medium

    CN115988218A

  • Virtualized video coding and decoding system and method, electronic equipment and storage medium

    CN116582438A

  • Virtualization method, device, board card and computer readable storage medium

    US20230111051A1

Cited By

  • Video encoding and decoding system and device, driving method, storage medium and electronic equipment

    CN119603453A

  • Virtualization coding method and equipment based on hardware video image coding and decoding system

    CN119653092A