Virtualization coding method and equipment based on hardware video image coding and decoding system

Abstract

The invention discloses a virtualized coding method and device based on a hardware video image coding and decoding system, the hardware video image coding and decoding system comprises a plurality of physically isolated video coding and decoding devices, each video coding and decoding device comprises a plurality of video coding and decoding cores, each video coding and decoding core is bound with a coding and decoding interface, and the coding and decoding interfaces are connected with the hardware video image coding and decoding system. In a memory used by the hardware video image coding and decoding system, an independent storage space is created for each virtual machine, and the method comprises the following steps: when a coding and decoding processing message sent by the virtual machine is received, sending the coding and decoding processing message to the virtual machine; the target video code determines an address of context data of a to-be-processed video session according to the equipment mapping relation and the video mapping relation; reading the context data of the to-be-processed video session from a target storage space of the storage space according to the address of the context data of the to-be-processed video session; and coding and decoding according to the context data of the to-be-processed video session, and writing a processing result back to the target storage space.

Description

Technical Field

[0001] The present invention belongs to the technical field of video coding and decoding, and in particular relates to a virtualized coding method and device based on a hardware video image coding and decoding system. Background Art

[0002] With the development of online video, video image data has increased dramatically. Usually, video encoding and decoding systems are deployed in the cloud to provide users with video encoding and decoding service networks.

[0003] At present, when the video encoding and decoding system deployed in the cloud provides video encoding and decoding services to multiple users at the same time, since each user may browse multiple videos, it is necessary to provide encoding and decoding services for multiple channels of videos to multiple users. How to improve the efficiency of video encoding and decoding in the cloud has become an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a virtualized encoding method and device based on a hardware video image encoding and decoding system, which can improve the efficiency of cloud video encoding and decoding.

[0005] According to a first aspect of the present invention, a virtualization encoding method based on a hardware video image encoding and decoding system is provided, comprising:

[0006] The method is applied to a hardware video image codec system, which includes a plurality of physically isolated video codec devices, each of which includes a plurality of video codec cores, and each of which is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine;

[0007] include:

[0008] Obtain the device mapping relationship between the virtual machine and the video codec device; obtain the video mapping relationship between each video codec session and the video codec core;

[0009] When receiving the codec processing message sent by the virtual machine, determining the target video codec device according to the device mapping relationship, and determining the target video codec core in the target video codec device according to the video mapping relationship;

[0010] The target video codec core determines the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; reads the context data of the video session to be processed from the target storage space of the storage space according to the address of the context data of the video session to be processed; performs codec processing according to the context data of the video session to be processed, and writes the processing result back to the target storage space.

[0011] According to a second aspect of the present invention, a virtualization encoding method based on a hardware video image codec system is provided, which is applied to a software end, wherein the software end is constructed with multiple virtual machines; each virtual machine includes at least one video codec session, and each video codec session has independent context data; the software end communicates with the hardware video image codec system, and the hardware video image codec system includes multiple physically isolated video codec devices, each video codec device includes multiple video codec cores, and each video codec core is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine; the method includes:

[0012] The virtual machine obtains the device mapping relationship between the virtual machine and the video codec device; obtains the video mapping relationship between each video codec session and the video codec core;

[0013] Determine the address of the context data of the to-be-processed video session according to the device mapping relationship and the video mapping relationship; write the context data of the to-be-processed video session into the address of the context data;

[0014] A codec processing message is sent to the video codec device bound in the hardware video image codec system, so that the video codec in the video codec device performs codec processing according to the context data.

[0015] According to a third aspect of the present invention, a virtualized encoding device based on a hardware video image codec system is provided, which is applied to the hardware video image codec system. The hardware video image codec system includes a plurality of physically isolated video codec devices, each of which includes a plurality of video codec cores, and each of which is bound to a codec interface. In the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine. The device includes:

[0016] A mapping relationship acquisition module is used to acquire a device mapping relationship between a virtual machine and a video codec device; and to acquire a video mapping relationship between each video codec session and a video codec core.

[0017] A video codec core determination module, configured to determine a target video codec device according to the device mapping relationship when receiving a codec processing message sent by the virtual machine, and determine a target video codec core in the target video codec device according to the video mapping relationship;

[0018] A codec processing module is used for the target video codec core to determine the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; read the context data of the video session to be processed from the target storage space of the storage space according to the address of the context data of the video session to be processed; perform codec processing according to the context data of the video session to be processed, and write the processing result back to the target storage space.

[0019] According to a fourth aspect of the present invention, a virtualized encoding device based on a hardware video image codec system is provided, which is applied to a software end, wherein the software end is constructed with multiple virtual machines; each virtual machine includes at least one video codec session, and each video codec session has independent context data; the software end communicates with the hardware video image codec system, and the hardware video image codec system includes multiple physically isolated video codec devices, each video codec device includes multiple video codec cores, and each video codec core is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine; the device includes:

[0020] A mapping acquisition module is used for the virtual machine to acquire the device mapping relationship between the virtual machine and the video codec device; and to acquire the video mapping relationship between each video codec session and the video codec core;

[0021] A context processing module, configured to determine an address of context data of a to-be-processed video session according to the device mapping relationship and the video mapping relationship; and write the context data of the to-be-processed video session into the address of the context data;

[0022] The codec triggering module is used to send a codec processing message to the video codec device bound to the hardware video image codec system, so that the video codec in the video codec device performs codec processing according to the context data.

[0023] According to a fifth aspect of the present invention, there is provided a virtualized encoding system based on a hardware video image encoding and decoding system, comprising:

[0024] Hardware video image encoding and decoding system;

[0025] at least one processor; and

[0026] a memory communicatively connected to the at least one processor; wherein,

[0027] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the virtualization encoding method based on the hardware video image encoding and decoding system described in any embodiment;

[0028] The hardware video image codec system includes a plurality of physically isolated video codec devices, each of which includes a plurality of video codec cores, and each of which is bound to a codec interface.

[0029] According to a sixth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the virtualization encoding method based on a hardware video image encoding and decoding system described in any embodiment of the present invention when executed.

[0030] The technical solution provided by the present invention provides a virtualized cloud video encoding and decoding method based on a hardware video image encoding and decoding system. The hardware video image encoding and decoding system includes multiple physically isolated video encoding and decoding devices, each video encoding and decoding device includes multiple video encoding and decoding cores, each video encoding and decoding core is bound to a codec interface, and a device mapping relationship between a virtual machine and a video encoding and decoding device is obtained; a video mapping relationship between each video encoding and decoding session and a video encoding and decoding core is obtained; when a codec processing message sent by the virtual machine is received, a target video encoding and decoding device is determined according to the device mapping relationship, and a target video encoding and decoding core in the target video encoding and decoding device is determined according to the video mapping relationship; the target video encoding and decoding core determines the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; the context data of the video session to be processed is read from the target storage space of the storage space according to the address of the context data of the video session to be processed; codec processing is performed according to the context data of the video session to be processed, and the processing result is written back to the target storage space. Since the hardware video image codec system itself has multiple physically isolated video codec devices, in the virtualized video codec scenario, it is possible to more effectively isolate each video codec device and enable multiple video codec devices to perform video codec synchronously. Based on the device mapping relationship and the video mapping relationship, the corresponding relationship between the video codec core in the video codec device and a certain recorded video in the virtual machine can be accurately determined, so that the video codec resources used by different virtual machines are isolated from each other, improving the utilization of video codec resources, and thus improving the efficiency of video codec.

[0031] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 It is a system architecture diagram of a virtualized encoding method based on a hardware video image encoding and decoding system provided by an embodiment of the present invention;

[0034] Figure 2 It is a flow chart of a virtualized encoding method based on a hardware video image encoding and decoding system provided by an embodiment of the present invention applied to a hardware video image encoding and decoding system;

[0035] Figure 3 It is a flowchart of a virtualized encoding method based on a hardware video image encoding and decoding system provided by an embodiment of the present invention applied to a software end;

[0036] Figure 4 It is a structural schematic diagram of a virtualized encoding device based on a hardware video image encoding and decoding system provided by an embodiment of the present invention applied to the hardware video image encoding and decoding system;

[0037] Figure 5 It is a structural schematic diagram of a virtualized encoding device based on a hardware video image encoding and decoding system provided by an embodiment of the present invention applied to a software end;

[0038] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] With the development of online video, video image data has increased dramatically. Usually, video encoding and decoding systems are deployed in the cloud to provide users with video encoding and decoding service networks.

[0042] At present, when a video codec system deployed in the cloud provides video codec services to multiple users at the same time, since each user may browse multiple videos, it is necessary to provide codec services for multiple channels of videos to multiple users. Codec services include encoding services and / or decoding services. In some scenarios, multi-user video codec services are implemented by using virtual machines created in the cloud, but since the video codec system is not used, video codec resources cannot be reasonably allocated, resulting in low utilization of video codec resources. How to improve the utilization of video codec resources in the cloud and thus improve the efficiency of video codec has become an urgent problem to be solved.

[0043] Figure 1 A system architecture diagram applicable to the virtualized encoding method based on the hardware video image encoding and decoding system provided in an embodiment of the present invention, the system architecture includes a software end, a hardware video image encoding and decoding system and a memory. The virtualized encoding method based on the hardware video image encoding and decoding system is set in the cloud to provide video encoding and decoding services for users.

[0044] The software end provided by the cloud includes multiple built virtual machines, each of which can provide video encoding and decoding services for one or a group of users. Users log in or create virtual machines by accessing the software end, and use the virtual machines to encode and decode videos. The number of virtual machines can be set according to demand, and the number of virtual machines changes with the increase or decrease of the number of users. Each virtual machine contains at least one video encoding and decoding session, and each video encoding and decoding session has independent context data.

[0045] The hardware video image codec system includes multiple physically isolated video codec devices, each of which contains multiple video codec cores, and each video codec core is bound to a codec interface. In the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine. In the memory, the storage space of each virtual machine is physically isolated, so the virtual machine writes the context data of the video to the matching storage space in the memory. This avoids the problem of memory data confusion caused by multiple virtual machines sharing memory.

[0046] Figure 2 The flowchart of the virtualized encoding method based on the hardware video image encoding and decoding system provided in the embodiment of the present invention is applicable to the case where the cloud provides virtualized video encoding and decoding services for users. The method can be implemented by hardware or software, for example, by configuring Figure 1 The electronic device implementation of the system architecture shown in FIG. Figure 2 As shown, the method can be applied to a hardware video image codec system, which includes multiple physically isolated video codec devices, each of which includes multiple video codec cores, and each of which is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine. The method includes:

[0047] Step S101: Obtain a device mapping relationship between a virtual machine and a video codec device; obtain a video mapping relationship between each video codec session and a video codec core.

[0048] The device mapping relationship between the virtual machine and the video codec device and the video mapping relationship between each video codec session and the video codec core can be pre-configured. The device mapping relationship can also be determined in real time based on the number relationship between the virtual machine and the video codec device. The video mapping relationship can be determined in real time based on the number relationship between the video codec session and the video codec core.

[0049] The software end includes multiple virtual machines, and the hardware video image codec system includes multiple video codec devices. The number of virtual machines and the number of video codec devices may be the same or different.

[0050] Optionally, obtaining the device mapping relationship between the virtual machine and the video codec device can be implemented in the following manner:

[0051] If the number of virtual machines is greater than the number of video codec devices, time-division multiplexing is performed on the video codec device resources to determine the device mapping relationship between the virtual machines and the video codec devices. ;

[0052] If the number of virtual machines is greater than the number of video codec devices, it means that the video codec device cannot provide video codec services for all virtual machines at the same time. At this time, the video coding device needs to be time-division multiplexed. That is, in the first multiplexing cycle, the target video coding device is matched to the first virtual machine. In the second multiplexing cycle, the target video coding device is matched to the second virtual machine. In different multiplexing cycles, a video codec device (that is, the target video codec device) provides video codec services for different virtual machines (such as the first virtual machine and the second virtual machine), thereby realizing time-division multiplexing of video codec device resources. In order to achieve the above effect, it is necessary to arrange a time-division multiplexing scheme so that when time-division multiplexing is performed, the video codec device is multiplexed according to the time-division multiplexing scheme. The above implementation method can multiplex the video coding device by time-division multiplexing when the number of video codec devices is less than the number of virtual machines, realize orderly video coding, and improve video coding efficiency.

[0053] Specifically, time division multiplexing of video codec device resources and determining a device mapping relationship between a virtual machine and a video codec device may be implemented as follows:

[0054] In a first multiplexing cycle, an association relationship between the first virtual machine and the target video encoding device is established so that the target video encoding device performs encoding and decoding processing according to the context of the to-be-processed video session of the video branch in the first virtual machine;

[0055] In the second multiplexing cycle, an association relationship between the second virtual machine and the target video encoding device is established so that the target video encoding device performs encoding and decoding processing according to the context of the to-be-processed video session of the video branch in the second virtual machine.

[0056] The first multiplexing cycle and the second multiplexing cycle may be two cyclic multiplexing cycles. By establishing virtual machines associated with target video encoding devices in different multiplexing cycles, the video encoding devices may be time-division multiplexed in an orderly manner, thereby improving the utilization rate of the video encoding and decoding devices.

[0057] Optionally, obtaining the device mapping relationship between the virtual machine and the video codec device can be implemented in the following manner:

[0058] If the number of virtual machines is less than or equal to the number of video codec devices, the video codec devices are allocated to the virtual machines in a spatial division multiplexing manner, and a device mapping relationship between the virtual machines and the video codec devices is determined.

[0059] If the number of virtual machines is less than or equal to the number of video codec devices, it means that the video codec device can provide video codec services for all virtual machines at the same time. Space division multiplexing is used to establish an association between the video codec device and the virtual machine in any cycle, and the video codec device corresponds to the virtual machine one by one. This prevents multiple virtual machines from transcoding through a video codec device at the same time, thereby improving the resource utilization of the video codec device.

[0060] On the basis of the above implementation, each virtual machine can simultaneously encode and decode multiple channels of video with the video codec device through multiple interfaces. The reason is that each video codec device includes multiple video codec cores. One video codec core receives context information of one channel of video through one interface. The number of video coding cores included in the video codec device can be increased or decreased according to demand. After determining the device mapping relationship between the video coding device and the virtual machine, a video mapping relationship between the video codec core and the virtual machine interface is established, and each interface corresponds to one channel of video.

[0061] Furthermore, obtaining the video mapping relationship between each video codec session and the video codec core can be implemented in the following manner:

[0062] If the number of video encoding sessions in the virtual machine increases, and if there are idle video codec cores in the video codec device bound to the virtual machine, a video mapping relationship between the video encoding sessions and the idle video codec cores is created;

[0063] If the video codec core in the video codec device bound to the virtual machine is not idle, a video mapping relationship between the video encoding session and the video codec core in the idle video codec device is created.

[0064] After the video mapping relationship is established, if the user adds a new video codec through the virtual machine, a new video codec core needs to be scheduled to provide video codec. There are two situations at this time: one is that the video codec core in the video codec device bound to the virtual machine is idle, and the other is that the video codec core in the video codec device bound to the virtual machine is not idle.

[0065] In the first case, a video mapping relationship between a video encoding session and an idle video codec core can be created. In the second case, a video codec core in an idle video codec device can be found. A video mapping relationship between a video codec core in an idle video codec device and a video encoding session in a virtual machine can be established.

[0066] Through the above implementation, when the number of video encoding and decoding sessions in the virtual machine increases, a new video mapping relationship can be reasonably established, thereby further improving the utilization rate of video encoding and decoding resources.

[0067] Step S102: when receiving the codec processing message sent by the virtual machine, determining the target video codec device according to the device mapping relationship, and determining the target video codec core in the target video codec device according to the video mapping relationship.

[0068] The video codec service can be started by the virtual machine sending a codec processing message to the hardware video image codec system or the video codec device. When the video codec device receives the encoder processing message sent by the virtual machine, the hardware video image codec system determines the target video codec device according to the device mapping relationship determined in step S101.

[0069] In one implementation, the above process may be executed by a processor in a hardware video image coding system. The processor determines a target video coding core corresponding to a video session in a virtual machine according to a device mapping relationship and a video mapping relationship.

[0070] In another implementation, the device mapping relationship and the video mapping relationship determined by the above process can be pre-configured to the virtual machine and the video codec device. When the virtual machine triggers the video codec service, the target video codec device is determined according to the mapped device mapping relationship. The interface used to send the video session that needs to be encoded and decoded is determined according to the video mapping relationship. The video codec core at the other end of the interface is determined through the interface and the video mapping relationship. The virtual machine can directly send an edge processing message to the video codec core.

[0071] Step S103: The target video codec core determines the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; reads the context data of the video session to be processed from the target storage space of the storage space according to the address of the context data of the video session to be processed; performs codec processing according to the context data of the video session to be processed, and writes the processing result back to the target storage space.

[0072] When the target video codec is encoding and decoding the video session to be processed, it needs to combine the context data of the video session to be processed for encoding and decoding. The context data is stored in the memory. The memory is as follows Figure 1 As shown, independent of the hardware video image codec system, an independent storage space is established for each virtual machine. The address of the context data accessed by the target video codec core is the read address in the storage space allocated by the virtual machine. Since each virtual machine has an independent storage space, the target video codec core only processes the context data in the independent storage space when encoding and decoding, without searching the entire memory space, and the address of the data matches the storage space, and its address length is less than the memory address length, which can improve the encoding and decoding efficiency of the video codec core.

[0073] After the target video codec core reads the context data of the video to be processed through the target storage space, it performs video editing and coding, and writes the processing result back to the target storage space.

[0074] The virtualization encoding method based on the hardware video image codec system provided by the present invention is applied to the hardware video image codec system, the hardware video image codec system includes a plurality of physically isolated video codec devices, each video codec device includes a plurality of video codec cores, each video codec core is bound to a codec interface, the method includes: obtaining a device mapping relationship between a virtual machine and a video codec device; obtaining a video mapping relationship between each video codec session and a video codec core; when receiving a codec processing message sent by the virtual machine, determining a target video codec device according to the device mapping relationship, and determining a target video codec core in the target video codec device according to the video mapping relationship; the target video codec core determines the address of context data of a to-be-processed video session according to the device mapping relationship and the video mapping relationship; reading the context data of the to-be-processed video session from a target storage space of the storage space according to the address of the context data of the to-be-processed video session; performing codec processing according to the context data of the to-be-processed video session, and writing the processing result back to the target storage space. Since the hardware video image codec system itself has multiple physically isolated video codec devices, in the virtualized video codec scenario, it is possible to more effectively isolate each video codec device and enable multiple video codec devices to perform video codec synchronously. Based on the device mapping relationship and the video mapping relationship, the corresponding relationship between the video codec core in the video codec device and a certain recorded video in the virtual machine can be accurately determined, so that the video codec resources used by different virtual machines are isolated from each other, improving the utilization of video codec resources, and thus improving the efficiency of video codec.

[0075] Figure 3 The flowchart of the virtualized encoding method based on the hardware video image encoding and decoding system provided in the embodiment of the present invention is applicable to the case where the cloud provides virtualized video encoding and decoding services for users. The method can be implemented by hardware or software, for example, by configuring Figure 1 The electronic device implementation of the system architecture shown in FIG. Figure 3As shown, the method can also be applied to the software side, the software side is constructed with multiple virtual machines; each virtual machine contains at least one video codec session, and each video codec session has independent context data; the software side communicates with the hardware video image codec system, the hardware video image codec system includes multiple physically isolated video codec devices, each video codec device contains multiple video codec cores, and each video codec core is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine. The method includes:

[0076] Step 201: The virtual machine obtains a device mapping relationship between the virtual machine and the video codec device; and obtains a video mapping relationship between each video codec session and a video codec core.

[0077] Optionally, obtaining a device mapping relationship between a virtual machine and a video codec device includes:

[0078] If the number of virtual machines is less than or equal to the number of video codec devices, the video codec devices are allocated to the virtual machines by spatial division multiplexing, and the device mapping relationship between the virtual machines and the video codec devices is determined. If the number of virtual machines is greater than the number of video codec devices, the video codec device resources are time-division multiplexed to determine the device mapping relationship between the virtual machines and the video codec devices.

[0079] Step 202: Determine the address of the context data of the to-be-processed video session according to the device mapping relationship and the video mapping relationship; and write the context data of the to-be-processed video session into the address of the context data.

[0080] Step 203: Send a codec processing message to the video codec device bound to the hardware video image codec system, so that the video codec in the video codec device performs codec processing according to the context data.

[0081] The virtualization encoding method based on the hardware video image codec system provided by the present invention is applied to the software end, and the software end is constructed with multiple virtual machines; each virtual machine includes at least one video codec session, and each video codec session has independent context data; the software end communicates with the hardware video image codec system, and the hardware video image codec system includes multiple physically isolated video codec devices, each video codec device includes multiple video codec cores, and each video codec core is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine. The virtual machine obtains the device mapping relationship between the virtual machine and the video codec device; obtains the video mapping relationship between each video codec session and the video codec core; determines the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; writes the context data of the video session to be processed into the address of the context data; sends a codec processing message to the video codec device bound to the hardware video image codec system, so that the video codec in the video codec device performs codec processing according to the context data. After obtaining the device mapping relationship and video mapping relationship, the software writes the context data of each video into the memory, and then sends the codec processing message to the video codec device bound to the hardware video image codec system. This can quickly and accurately start the codec of multiple videos, enable different video codec cores to synchronously encode and decode multiple videos, and improve the efficiency of video codec.

[0082] Figure 4 A structural schematic diagram of a virtualized encoding device based on a hardware video image codec system provided in an embodiment of the present invention, wherein the virtualized encoding device based on a hardware video image codec system is used to implement the above-mentioned virtualized encoding method based on a hardware video image codec system, and the application is based on a hardware video image codec system, wherein the hardware video image codec system includes a plurality of physically isolated video codec devices, each of which includes a plurality of video codec cores, and each of which is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine; the device includes: a mapping relationship acquisition module 31, a video codec core determination module 32, and a codec processing module 33.

[0083] A mapping relationship acquisition module 31 is used to acquire a device mapping relationship between a virtual machine and a video codec device; and to acquire a video mapping relationship between each video codec session and a video codec core.

[0084] The video codec core determination module 32 is used to determine the target video codec device according to the device mapping relationship when receiving the codec processing message sent by the virtual machine, and determine the target video codec core in the target video codec device according to the video mapping relationship;

[0085] The codec processing module 33 is used for the target video codec core to determine the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; read the context data of the video session to be processed from the target storage space of the storage space according to the address of the context data of the video session to be processed; perform codec processing according to the context data of the video session to be processed, and write the processing result back to the target storage space.

[0086] Optionally, based on the above implementation, the mapping relationship acquisition module 31 is used to:

[0087] If the number of virtual machines is less than or equal to the number of video codec devices, the video codec devices are allocated to the virtual machines in a spatial division multiplexing manner, and a device mapping relationship between the virtual machines and the video codec devices is determined.

[0088] Optionally, based on the above implementation, the mapping relationship acquisition module 31 is used to:

[0089] If the number of video encoding sessions in the virtual machine increases, and if there are idle video codec cores in the video codec device bound to the virtual machine, a video mapping relationship between the video encoding sessions and the idle video codec cores is created;

[0090] If the video codec core in the video codec device bound to the virtual machine is not idle, a video mapping relationship between the video encoding session and the video codec core in the idle video codec device is created.

[0091] Optionally, based on the above implementation, the mapping relationship acquisition module 31 is used to:

[0092] If the number of virtual machines is greater than the number of video codec devices, time division multiplexing is performed on the video codec device resources to determine a device mapping relationship between the virtual machines and the video codec devices.

[0093] Optionally, based on the above implementation, the mapping relationship acquisition module 31 is used to:

[0094] In a first multiplexing cycle, an association relationship between the first virtual machine and the target video encoding device is established so that the target video encoding device performs encoding and decoding processing according to the context of the to-be-processed video session of the video branch in the first virtual machine;

[0095] In the second multiplexing cycle, an association relationship between the second virtual machine and the target video encoding device is established so that the target video encoding device performs encoding and decoding processing according to the context of the to-be-processed video session of the video branch in the second virtual machine.

[0096] The virtualization encoding method based on the hardware video image codec system provided by the present invention is applied to the hardware video image codec system, the hardware video image codec system includes a plurality of physically isolated video codec devices, each video codec device includes a plurality of video codec cores, each video codec core is bound to a codec interface, the method includes: a mapping relationship acquisition module 31, used to acquire the device mapping relationship between the virtual machine and the video codec device; acquire the video mapping relationship between each video codec session and the video codec core; a video codec core determination module 32, used to determine the target video codec device according to the device mapping relationship when receiving the codec processing message sent by the virtual machine, and determine the target video codec core in the target video codec device according to the video mapping relationship; a codec processing module 33, used for the target video codec core to determine the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; read the context data of the video session to be processed from the target storage space of the storage space according to the address of the context data of the video session to be processed; perform codec processing according to the context data of the video session to be processed, and write the processing result back to the target storage space. Since the hardware video image codec system itself has multiple physically isolated video codec devices, in the virtualized video codec scenario, it is possible to more effectively isolate each video codec device and enable multiple video codec devices to perform video codec synchronously. Based on the device mapping relationship and the video mapping relationship, the corresponding relationship between the video codec core in the video codec device and a certain recorded video in the virtual machine can be accurately determined, so that the video codec resources used by different virtual machines are isolated from each other, improving the utilization of video codec resources, and thus improving the efficiency of video codec.

[0097] Figure 5A structural schematic diagram of a virtualization encoding device based on a hardware video image codec system provided in an embodiment of the present invention, the virtualization encoding device based on a hardware video image codec system is used to implement the above-mentioned virtualization encoding method based on a hardware video image codec system, an application software end, the software end is constructed with multiple virtual machines; each virtual machine includes at least one video codec session, and each video codec session has independent context data; the software end communicates with the hardware video image codec system, the hardware video image codec system includes multiple physically isolated video codec devices, each video codec device includes multiple video codec cores, and each video codec core is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine; the device includes: a mapping acquisition module 41, a context processing module 42 and a codec trigger module 43.

[0098] A mapping acquisition module 41 is used for the virtual machine to acquire a device mapping relationship between the virtual machine and the video codec device; and to acquire a video mapping relationship between each video codec session and the video codec core;

[0099] A context processing module 42 is used to determine the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; and write the context data of the video session to be processed into the address of the context data;

[0100] The codec triggering module 43 is used to send a codec processing message to the video codec device bound to the hardware video image codec system, so that the video codec in the video codec device performs codec processing according to the context data.

[0101] Optionally, based on the above implementation, the mapping acquisition module 41 is used to:

[0102] If the number of virtual machines is less than or equal to the number of video codec devices, the video codec devices are allocated to the virtual machines by spatial division multiplexing, and a device mapping relationship between the virtual machines and the video codec devices is determined;

[0103] If the number of virtual machines is greater than the number of video codec devices, time division multiplexing is performed on the video codec device resources to determine a device mapping relationship between the virtual machines and the video codec devices.

[0104] The virtualization encoding device based on the hardware video image codec system provided by the present invention is applied to the software end, and the software end is constructed with multiple virtual machines; each virtual machine includes at least one video codec session, and each video codec session has independent context data; the software end communicates with the hardware video image codec system, and the hardware video image codec system includes multiple physically isolated video codec devices, each video codec device includes multiple video codec cores, and each video codec core is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine. A mapping acquisition module 41 is used for the virtual machine to obtain the device mapping relationship between the virtual machine and the video codec device; to obtain the video mapping relationship between each video codec session and the video codec core; a context processing module 42 is used to determine the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; write the context data of the video session to be processed into the address of the context data; a codec trigger module 43 is used to send a codec processing message to the video codec device bound in the hardware video image codec system, so that the video codec in the video codec device performs codec processing according to the context data. After obtaining the device mapping relationship and video mapping relationship, the software writes the context data of each video into the memory, and then sends the codec processing message to the video codec device bound to the hardware video image codec system. This can quickly and accurately start the codec of multiple videos, enable different video codec cores to synchronously encode and decode multiple videos, and improve the efficiency of video codec.

[0105] The virtualization encoding device based on the hardware video image encoding and decoding system provided in the embodiment of the present invention can execute the virtualization encoding method based on the hardware video image encoding and decoding system provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0106] Figure 6 6 is a schematic diagram of a virtualized encoding system based on a hardware video image encoding and decoding system provided by an embodiment of the present invention. The electronic device 6 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0107] like Figure 6As shown, the virtualized encoding system 10 based on the hardware video image codec system includes the hardware video image codec system, at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In RAM 13, various programs and data required for the operation of the virtualized encoding system 10 based on the hardware video image codec system can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14. The hardware video image codec system includes a plurality of physically isolated video codec devices, each of which includes a plurality of video codec cores, and each video codec core is bound to a codec interface.

[0108] Exemplarily, in the above implementation, the memory, the software end and the hardware video image encoding and decoding system may be respectively located in three boards in the blade server, and the blade server is used to provide cloud encoding and decoding services.

[0109] Multiple components in the virtualized encoding system 10 based on the hardware video image codec system are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the virtualized encoding system 10 based on the hardware video image codec system to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0110] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a virtualized encoding method based on a hardware video image encoding and decoding system.

[0111] In some embodiments, the virtualization encoding method based on the hardware video image codec system can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the virtualization encoding system 10 based on the hardware video image codec system via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the virtualization encoding method based on the hardware video image codec system described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the virtualization encoding method based on the hardware video image codec system in any other appropriate manner (for example, by means of firmware).

[0112] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] The computer program for implementing the virtualized encoding method based on the hardware video image encoding and decoding system of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or the block diagram are implemented. The computer program can be executed completely on the machine, partially on the machine, as an independent software package, partially on the machine and partially on a remote machine, or completely on a remote machine or server.

[0114] The embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions. Optionally, the computer instructions can be used to enable a processor to execute a virtualized encoding method based on a hardware video image codec system, the method being applied to a hardware video image codec system, the hardware video image codec system comprising a plurality of physically isolated video codec devices, each video codec device comprising a plurality of video codec cores, each video codec core being bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine; the method comprising:

[0115] Obtain the device mapping relationship between the virtual machine and the video codec device; obtain the video mapping relationship between each video codec session and the video codec core;

[0116] When receiving the codec processing message sent by the virtual machine, determining the target video codec device according to the device mapping relationship, and determining the target video codec core in the target video codec device according to the video mapping relationship;

[0117] The target video codec core determines the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; reads the context data of the video session to be processed from the target storage space of the storage space according to the address of the context data of the video session to be processed; performs codec processing according to the context data of the video session to be processed, and writes the processing result back to the target storage space.

[0118] Based on the above implementation, optionally, obtaining a device mapping relationship between a virtual machine and a video codec device includes:

[0119] If the number of virtual machines is less than or equal to the number of video codec devices, the video codec devices are allocated to the virtual machines in a spatial division multiplexing manner, and a device mapping relationship between the virtual machines and the video codec devices is determined.

[0120] Based on the above implementation, optionally, obtaining a video mapping relationship between each video codec session and a video codec core includes:

[0121] If the number of video encoding sessions in the virtual machine increases, and if there are idle video codec cores in the video codec device bound to the virtual machine, a video mapping relationship between the video encoding sessions and the idle video codec cores is created;

[0122] If the video codec core in the video codec device bound to the virtual machine is not idle, a video mapping relationship between the video encoding session and the video codec core in the idle video codec device is created.

[0123] Based on the above implementation, optionally, obtaining a device mapping relationship between a virtual machine and a video codec device includes:

[0124] If the number of virtual machines is greater than the number of video codec devices, time division multiplexing is performed on the video codec device resources to determine a device mapping relationship between the virtual machines and the video codec devices.

[0125] On the basis of the above implementation, optionally, time division multiplexing is performed on the video codec device resources to determine the device mapping relationship between the virtual machine and the video codec device, including:

[0126] In a first multiplexing cycle, an association relationship between the first virtual machine and the target video encoding device is established so that the target video encoding device performs encoding and decoding processing according to the context of the to-be-processed video session of the video branch in the first virtual machine;

[0127] In the second multiplexing cycle, an association relationship between the second virtual machine and the target video encoding device is established so that the target video encoding device performs encoding and decoding processing according to the context of the to-be-processed video session of the video branch in the second virtual machine.

[0128] The embodiment of the present invention also provides a computer-readable storage medium, which stores computer instructions. Optionally, the computer instructions can be used to enable a processor to execute a virtualized encoding method based on a hardware video image codec system, which is applied to a software end, and the software end is constructed with multiple virtual machines; each virtual machine includes at least one video codec session, and each video codec session has independent context data; the software end communicates with the hardware video image codec system, and the hardware video image codec system includes multiple physically isolated video codec devices, each video codec device includes multiple video codec cores, and each video codec core is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine; the method includes:

[0129] The virtual machine obtains the device mapping relationship between the virtual machine and the video codec device; obtains the video mapping relationship between each video codec session and the video codec core;

[0130] Determine the address of the context data of the to-be-processed video session according to the device mapping relationship and the video mapping relationship; write the context data of the to-be-processed video session into the address of the context data;

[0131] A codec processing message is sent to the video codec device bound in the hardware video image codec system, so that the video codec in the video codec device performs codec processing according to the context data.

[0132] Based on the above implementation, optionally, obtaining a device mapping relationship between a virtual machine and a video codec device includes:

[0133] If the number of virtual machines is less than or equal to the number of video codec devices, the video codec devices are allocated to the virtual machines by spatial division multiplexing, and a device mapping relationship between the virtual machines and the video codec devices is determined;

[0134] If the number of virtual machines is greater than the number of video codec devices, time division multiplexing is performed on the video codec device resources to determine a device mapping relationship between the virtual machines and the video codec devices.

[0135] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0137] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0138] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0139] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0140] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A virtualized encoding method based on a hardware video image encoding and decoding system, characterized in that: Applied to a hardware video image codec system, the hardware video image codec system includes a plurality of physically isolated video codec devices, each video codec device includes a plurality of video codec cores, and each video codec core is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine; the method includes: Obtain the device mapping relationship between the virtual machine and the video codec device; obtain the video mapping relationship between each video codec session and the video codec core; When receiving the codec processing message sent by the virtual machine, determining the target video codec device according to the device mapping relationship, and determining the target video codec core in the target video codec device according to the video mapping relationship; The target video codec core determines the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; reads the context data of the video session to be processed from the target storage space of the storage space according to the address of the context data of the video session to be processed; performs codec processing according to the context data of the video session to be processed, and writes the processing result back to the target storage space.

2. The method according to claim 1, characterized in that Get the device mapping relationship between the virtual machine and the video codec device, including: If the number of virtual machines is less than or equal to the number of video codec devices, the video codec devices are allocated to the virtual machines in a spatial division multiplexing manner, and a device mapping relationship between the virtual machines and the video codec devices is determined.

3. The method according to claim 2, characterized in that Get the video mapping relationship between each video codec session and the video codec core, including: If the number of video encoding sessions in the virtual machine increases, and if there are idle video codec cores in the video codec device bound to the virtual machine, a video mapping relationship between the video encoding sessions and the idle video codec cores is created; If the video codec core in the video codec device bound to the virtual machine is not idle, a video mapping relationship between the video encoding session and the video codec core in the idle video codec device is created.

4. The method according to claim 1, characterized in that Get the device mapping relationship between the virtual machine and the video codec device, including: If the number of virtual machines is greater than the number of video codec devices, time division multiplexing is performed on the video codec device resources to determine a device mapping relationship between the virtual machines and the video codec devices.

5. The method according to claim 4, characterized in that Time-division multiplexing of video codec device resources is performed to determine the device mapping relationship between the virtual machine and the video codec device, including: In a first multiplexing cycle, an association relationship between the first virtual machine and the target video encoding device is established so that the target video encoding device performs encoding and decoding processing according to the context of the to-be-processed video session of the video branch in the first virtual machine; In the second multiplexing cycle, an association relationship between the second virtual machine and the target video encoding device is established so that the target video encoding device performs encoding and decoding processing according to the context of the to-be-processed video session of the video branch in the second virtual machine.

6. A virtualized encoding method based on a hardware video image encoding and decoding system, characterized in that: Applied to a software end, the software end is constructed with multiple virtual machines; each virtual machine includes at least one video codec session, and each video codec session has independent context data; the software end communicates with a hardware video image codec system, and the hardware video image codec system includes multiple physically isolated video codec devices, each video codec device includes multiple video codec cores, and each video codec core is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine; the method includes: The virtual machine obtains the device mapping relationship between the virtual machine and the video codec device; obtains the video mapping relationship between each video codec session and the video codec core; Determine the address of the context data of the to-be-processed video session according to the device mapping relationship and the video mapping relationship; write the context data of the to-be-processed video session into the address of the context data; A codec processing message is sent to the video codec device bound in the hardware video image codec system, so that the video codec in the video codec device performs codec processing according to the context data.

7. The method according to claim 6, characterized in that Get the device mapping relationship between the virtual machine and the video codec device, including: If the number of virtual machines is less than or equal to the number of video codec devices, the video codec devices are allocated to the virtual machines by spatial division multiplexing, and a device mapping relationship between the virtual machines and the video codec devices is determined; If the number of virtual machines is greater than the number of video codec devices, time division multiplexing is performed on the video codec device resources to determine a device mapping relationship between the virtual machines and the video codec devices.

8. A virtualized encoding device based on a hardware video image encoding and decoding system, characterized in that: Applied to a hardware video image codec system, the hardware video image codec system includes a plurality of physically isolated video codec devices, each of which includes a plurality of video codec cores, and each of which is bound to a codec interface; in the memory used by the hardware video image codec system, an independent storage space is created for each virtual machine; the device includes: A mapping relationship acquisition module is used to acquire a device mapping relationship between a virtual machine and a video codec device; and to acquire a video mapping relationship between each video codec session and a video codec core. A video codec core determination module, configured to determine a target video codec device according to the device mapping relationship when receiving a codec processing message sent by the virtual machine, and determine a target video codec core in the target video codec device according to the video mapping relationship; A codec processing module is used for the target video codec core to determine the address of the context data of the video session to be processed according to the device mapping relationship and the video mapping relationship; read the context data of the video session to be processed from the target storage space of the storage space according to the address of the context data of the video session to be processed; perform codec processing according to the context data of the video session to be processed, and write the processing result back to the target storage space.

9. A virtualized encoding system based on a hardware video image encoding and decoding system, characterized in that: include: Hardware video image encoding and decoding system; at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the virtualization encoding method based on the hardware video image encoding and decoding system according to any one of claims 1 to 7; The hardware video image codec system includes a plurality of physically isolated video codec devices, each of which includes a plurality of video codec cores, and each of which is bound to a codec interface.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the virtualization encoding method based on a hardware video image encoding and decoding system according to any one of claims 1 to 7 when executed.

Images