Video data transmission system, method, device, equipment and medium

Abstract

The invention relates to a video data transmission system, method and device, equipment and a medium. The system comprises a VPU, a GPU and a host, the VPU is used for setting configuration parameters of the VPU, and the configuration parameters are used for configuring a first mapping relation between a specified memory in the VPU and a PCIe memory space; the host is used for selecting a part of the first sub-memory from the VPU specified memory as a P2P memory, and establishing a second mapping relationship between the first sub-memory and the host memory; the GPU is used for setting configuration parameters of the GPU, and the configuration parameters are used for configuring a third mapping relation between a specified memory in the GPU and the PCIe memory space; the host is used for selecting a part of the second sub-memory from the GPU specified memory as a P2P memory, and establishing a fourth mapping relationship between the second sub-memory and the host memory; and the host is used for realizing specified video data transmission between the first sub-memory and the second sub-memory according to the third mapping relationship or the fourth mapping relationship. CPU intervention is not needed, so that the data transmission efficiency is improved, the load of the central processing unit is reduced, and the system performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of video transmission technology, and in particular to a video data transmission system, method, device, equipment and medium. Background Art

[0002] Currently, both enterprises and individual consumers require higher-quality video data, and the improvement of video resolution, frame rate and real-time performance requires higher video encoding and decoding efficiency and faster data transmission speed. The traditional method of using CPU to complete video encoding and decoding consumes a lot of computing power and is costly when processing high-resolution videos. In order to reduce the CPU load and support multi-channel high-definition video encoding and decoding, a dedicated video processing unit (VPU) was introduced. As a new hardware device on the server, VPU needs to realize the functions of communication and data transmission with other devices on the motherboard. VPU uses NVMe io to realize inter-device communication. When other devices other than CPU are not involved, data transmission only occurs between VPU memory and system memory, and the DMA engine in VPU completes the data transmission between the two. If the video data needs to be processed more complexly, the current AI image processing such as face recognition and motion tracking usually requires GPU intervention. The traditional data transmission method will be limited by CPU load and memory bandwidth and cannot meet the needs of processing multiple channels of high frame rate videos at the same time. Traditional data transmission between devices requires the intervention of CPU and host memory. This process will increase the CPU load and become a performance bottleneck in scenarios where a large amount of data transmission is required, resulting in the hardware performance of VPU and GPU cannot be fully utilized. Summary of the invention

[0003] The purpose of this application is to provide a video data transmission system, method, device, equipment and medium, which can effectively improve the data processing efficiency of VPU and GPU.

[0004] According to a first aspect of an embodiment of the present application, a video data transmission system is provided, the system comprising: a VPU, a GPU and a host.

[0005] The VPU is used to set configuration parameters of the VPU, and the configuration parameters are used to configure a first mapping relationship between a specified memory in the VPU and a PCIe memory space;

[0006] The host is used to select a part of the first sub-memory from the designated memory of the VPU as the P2P memory, and establish a second mapping relationship between the first sub-memory and the host memory;

[0007] The GPU is used to set configuration parameters of the GPU, and the configuration parameters are used to configure a third mapping relationship between a specified memory in the GPU and a PCIe memory space;

[0008] The host is used to select a part of the second sub-memory from the designated memory of the GPU as the P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory;

[0009] The host is used to implement designated video data transmission between the first sub-memory and the second sub-memory according to the third mapping relationship or the fourth mapping relationship.

[0010] Optionally, the VPU is further used to update an open flag bit O_RDWR in firmware, and a first corresponding relationship between the flag bit and the designated memory.

[0011] Optionally, the host is used to determine the starting address and storage space size corresponding to the first sub-memory by driving a PCIe BAR configured for the VPU;

[0012] It is also used to determine the starting address and storage space size corresponding to the first sub-memory by driving the PCIe BAR configured for the GPU.

[0013] Optionally, the first sub-memory and the second sub-memory store video data of a specified type, including: one or more of YUV raw video data and RGB raw video data.

[0014] Optionally, when the host sends a transmission request for the designated video data to the VPU, the host obtains a memory address of the first sub-memory from the VPU;

[0015] When the host sends a data transmission request for the designated video to the GPU, the host obtains a memory address of the second sub-memory from the GPU;

[0016] The host generates data transmission information according to the memory address of the first sub-memory and the memory address of the second sub-memory;

[0017] The PCIe transfers the designated video data in the first sub-memory to the second sub-memory according to the data transmission information.

[0018] According to a second aspect of an embodiment of the present application, a video data transmission method is provided, which is applied to a VPU, and the method includes:

[0019] In response to the host sending a data transmission request for the specified video, sending a pre-configured memory address of a first sub-memory to the host, so that the host uses the memory address of the first sub-memory and the memory address of the second sub-memory of the GPU to send data transmission information to PCIe;

[0020] In response to the data transmission instruction sent by the PCIe, the specified video data in the first sub-memory is sent to the second sub-memory of the GPU.

[0021] Optionally, before sending the preconfigured memory address of the first sub-memory to the host, the method further includes:

[0022] Setting configuration parameters of a first mapping relationship between a specified memory in the VPU and a PCIe memory space;

[0023] A portion of the first sub-memory is selected from the designated memory as a P2P memory according to the first mapping relationship, so as to establish a second mapping relationship between the first sub-memory and the host memory.

[0024] According to a third aspect of an embodiment of the present application, a video data transmission method is provided, which is applied to a GPU, and the method includes:

[0025] In response to the host sending a data transmission request for the designated video, sending a memory address of a preconfigured second sub-memory to the host;

[0026] The PCIe transfers the designated video data in the first sub-memory to the second sub-memory according to the data transmission information.

[0027] Optionally, before sending the pre-configured memory address of the second sub-memory to the host, the method further includes:

[0028] Set the configuration parameters of the third mapping relationship between the specified memory in the GPU and the PCIe memory space;

[0029] A portion of the second sub-memory is selected from the designated memory as the P2P memory according to the third mapping relationship, so as to establish a fourth mapping relationship between the second sub-memory and the host memory.

[0030] According to a third aspect of an embodiment of the present application, a video data transmission method is provided, which is applied to a host, and the method includes:

[0031] Selecting a portion of the first sub-memory from the VPU designated memory as the P2P memory, and establishing a second mapping relationship between the first sub-memory and the host memory; wherein the VPU designated memory is a first mapping relationship between the VPU and the PCIe memory space configured by setting configuration parameters;

[0032] Select a portion of the second sub-memory from the designated memory of the GPU as the P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory; wherein the designated memory of the GPU is the third mapping relationship between the GPU and the PCIe memory space by setting configuration parameters;

[0033] The designated video data transmission between the first sub-memory and the second sub-memory is implemented according to the third mapping relationship or the fourth mapping relationship.

[0034] According to a fifth aspect of an embodiment of the present application, a video data transmission device is provided, which is applied to a VPU, and the device includes:

[0035] A first sending module, configured to send a pre-configured memory address of a first sub-memory to the host in response to the host sending a data transmission request for the specified video data, so that the host uses the memory address of the first sub-memory and the memory address of the second sub-memory of the GPU to send data transmission information to the PCIe;

[0036] The second sending module sends the specified video data in the first sub-memory to the second sub-memory of the GPU in response to the data transmission instruction sent by the PCIe.

[0037] According to a sixth aspect of an embodiment of the present application, a video data transmission device is provided, which is applied to a GPU, and the device includes:

[0038] A first sending module, configured to send a memory address of a pre-configured second sub-memory to the host in response to the host sending a data transmission request for the designated video;

[0039] A second sending module is used for the PCIe to transfer the specified video data in the first sub-memory to the second sub-memory according to the data transmission information.

[0040] According to a seventh aspect of an embodiment of the present application, a video data transmission device is provided, which is applied to a host, and the device includes:

[0041] A first relationship establishment module is used to select a portion of the first sub-memory from the VPU designated memory as the P2P memory, and establish a second mapping relationship between the first sub-memory and the host memory; wherein the VPU designated memory is a first mapping relationship between the VPU and the PCIe memory space configured by setting configuration parameters;

[0042] A second relationship establishment module is used to select a part of the second sub-memory from the designated memory of the GPU as the P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory; wherein the designated memory of the GPU is the third mapping relationship between the GPU and the PCIe memory space by setting configuration parameters;

[0043] A data transmission module is used to implement the specified video data transmission between the first sub-memory and the second sub-memory according to the third mapping relationship or the fourth mapping relationship.

[0044] According to an eighth aspect of an embodiment of the present application, there is provided an electronic device, comprising a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the above method.

[0045] According to a ninth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the executable computer program in the storage medium is executed by a processor, the above method can be implemented.

[0046] Compared with the prior art, the beneficial effect of the present application is that: the system includes: VPU, GPU and host, VPU is used to set the configuration parameters of VPU, the configuration parameters are used to configure the first mapping relationship between the specified memory in VPU and PCIe memory space; the host is used to select part of the first sub-memory from the VPU specified memory as P2P memory, and establish a second mapping relationship between the first sub-memory and the host memory; GPU is used to set the configuration parameters of GPU, the configuration parameters are used to configure the third mapping relationship between the specified memory in GPU and PCIe memory space; the host is used to select part of the second sub-memory from the GPU specified memory as P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory; the host is used to realize the specified video data transmission between the first sub-memory and the second sub-memory according to the third mapping relationship or the fourth mapping relationship. In summary, the technical solution provided by the present application directly transmits data between the video processing hardware device (VPU) and other image processing hardware devices (GPU) through PCIe without the intervention of the central processing unit (CPU), thereby improving data transmission efficiency, reducing the load of the central processing unit, and improving system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of the structure of a video data transmission system provided in an embodiment of the present application.

[0048] Figure 2 A flowchart of a video data transmission method provided in an embodiment of the present application.

[0049] Figure 3 A flowchart of another video data transmission method provided in an embodiment of the present application.

[0050] Figure 4 A flowchart of another video data transmission method provided in an embodiment of the present application.

[0051] Figure 5 is a block diagram of a video data transmission device according to an exemplary embodiment.

[0052] Figure 6is a block diagram of another video data transmission device according to an exemplary embodiment.

[0053] Figure 7 is a block diagram of yet another video data transmission device according to an exemplary embodiment.

[0054] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0055] Unless otherwise defined, the technical terms or scientific terms used in this specification and claims shall have the usual meanings understood by persons with ordinary skills in the technical field to which the invention belongs. The specific embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be noted that in the specific description of these embodiments, in order to provide a concise description, it is impossible for this specification to provide a detailed description of all the features of the actual embodiments. Without departing from the spirit and scope of the present invention, those skilled in the art may modify and replace the embodiments of the present invention, and the resulting embodiments are also within the scope of protection of the present invention.

[0056] Terminology explanation:

[0057] VPU (Video Processing Unit): VPU is a processor specifically used to process video data. VPU is mainly used to perform tasks such as video encoding, decoding, scaling, filtering, etc. It can improve the efficiency and quality of video processing. VPU is widely used in video conferencing, digital television, game development, virtual reality and other fields. In these scenarios, VPU can provide smoother video playback and better image quality. The performance of VPU depends on its design and implementation. A high-performance VPU can process multiple video streams simultaneously and support high-resolution and high-frame-rate video processing. At the same time, the power consumption of VPU also depends on its design and workload. The power consumption management of VPU needs to be considered when designing low-power devices.

[0058] GPU (Graphic Processing Unit): GPU, also known as display core, visual processor, display chip, is a microprocessor that specializes in image computing on personal computers, workstations, game consoles, and some mobile devices (such as tablets, smartphones, etc.). In addition to traditional graphics processing tasks, GPU is also widely used in scientific computing, deep learning and other fields. For example, GPU can be used to accelerate the training and reasoning of neural networks, and is very efficient when processing large amounts of data.

[0059] P2P: (Peer-to-Peer) peer-to-peer network or point-to-peer network. In this network architecture, networked computers are independent and equal, without primary and secondary distinction. Instead of connecting to the server to download data, they directly connect to other users' computers in the network to achieve resource and information sharing. The P2P system has a high degree of fault tolerance and very good scalability. It can quickly distribute data and aggregate resources. It shows advantages in large-scale concurrent services and can also reduce the cost of network construction.

[0060] PCIe: (Peripheral Component Interconnect Express) is a high-speed serial computer expansion bus standard. It is mainly used to connect various components inside the computer, such as graphics cards, network cards, and sound cards, so that they (especially with chipsets and processors) can efficiently transmit data and work together.

[0061] like Figure 1 A schematic diagram of the structure of a video data transmission system provided in an embodiment of the present application. Figure 1 As can be seen in the figure, the system mainly includes VPU, GPU, host (CPU) and PCIe.

[0062] The VPU is used to set configuration parameters of the VPU, and the configuration parameters are used to configure a first mapping relationship between a specified memory in the VPU and a PCIe memory space. The host is used to select a portion of a first sub-memory from the specified memory of the VPU as a P2P memory, and establish a second mapping relationship between the first sub-memory and the host memory.

[0063] The GPU is used to set configuration parameters of the GPU, and the configuration parameters are used to configure a third mapping relationship between a specified memory in the GPU and a PCIe memory space. The host is used to select a portion of a second sub-memory from the specified memory of the GPU as a P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory.

[0064] The host is used to implement designated video data transmission between the first sub-memory and the second sub-memory according to the third mapping relationship or the fourth mapping relationship.

[0065] The following will explain the relevant technical implementation process and principles in the system.

[0066] VPU configuration parameter setting details: VPU configuration parameters include but are not limited to PCIe BAR (Base Address Register) settings, which define the specific areas where VPU memory is mapped to PCIe address space. Configuration parameters should also include protection properties of VPU memory, such as whether to allow reading and writing, whether to enable caching, etc., to ensure the security and efficiency of data transmission.

[0067] The host selects the P2P memory process: When the host selects the first sub-memory in the VPU as the P2P memory, it needs to consider factors such as the memory alignment requirements, size, and whether it is occupied by other processes. Similarly, when selecting the second sub-memory in the GPU, similar considerations need to be followed. The host interacts with the VPU and GPU through the driver to obtain their memory layout and status information to make an appropriate selection.

[0068] Specific steps to establish a mapping relationship: When establishing the first mapping relationship between the VPU specified memory and the PCIe memory space, the host needs to send a configuration command to the VPU and wait for the VPU to confirm that the mapping is successful. Similarly, similar steps are required to establish the third mapping relationship between the GPU specified memory and the PCIe memory space. When establishing the second and fourth mapping relationships, the host needs to use the memory management function of the operating system to map the selected P2P memory area to the host address space so that the user space program can access it.

[0069] Implementation method of data transmission: When data transmission between P2P memories is implemented according to the third mapping relationship or the fourth mapping relationship, the host can start the transmission process by sending a DMA command. The DMA command should contain information such as the source address (the starting address of the first sub-memory or the second sub-memory), the target address (the starting address of another P2P memory), the transfer size, and the transfer direction. During the transmission process, the DMA engine of the VPU and GPU will be responsible for the actual movement of the data without the intervention of the CPU.

[0070] Error handling and status monitoring: During data transmission, the host needs to monitor the status of the VPU and GPU, as well as the status of the PCIe bus to ensure smooth transmission. If an error occurs (such as transmission timeout, data verification failure, etc.), the host needs to take appropriate error handling measures, such as retrying the transmission, recording an error log, or notifying the user space program.

[0071] Security and isolation: To ensure the security and isolation of data transmission, the host needs to set appropriate access rights and protection mechanisms for the P2P memory areas of the VPU and GPU. This can be achieved through the memory management function of the operating system, the firmware of the VPU and GPU, and the security policies in the driver.

[0072] The VPU is used to set its configuration parameters, which specifically define the first mapping relationship between the specified memory in the VPU and the PCIe memory space. These configuration parameters include but are not limited to the setting of PCIe BAR and the configuration of memory protection attributes. The host is responsible for selecting part of the first sub-memory from the specified memory of the VPU as P2P memory, and interacting with the VPU through a driver to establish a second mapping relationship between the first sub-memory and the host memory. During the selection process, the host will consider factors such as the alignment requirements, size, and occupancy of the memory. Similarly, the GPU is also used to set its configuration parameters to define the third mapping relationship between the specified memory in the GPU and the PCIe memory space.

[0073] The host will also select part of the second sub-memory from the designated memory of the GPU as the P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory. This step also needs to consider factors such as memory alignment, size, and occupancy. After these mapping relationships are established, the host can implement the designated video data transmission between the first sub-memory and the second sub-memory according to the third mapping relationship or the fourth mapping relationship (depending on the direction of data transmission). This transmission process is initiated by sending a DMA command, and the DMA engines of the VPU and GPU are responsible for the actual movement of data. During the entire data transmission process, the host needs to monitor the status of the VPU and GPU and the status of the PCIe bus to ensure smooth transmission. At the same time, in order to ensure the security and isolation of data transmission, the host also needs to set appropriate access rights and protection mechanisms for the P2P memory areas of the VPU and GPU.

[0074] By establishing the third mapping relationship and the fourth mapping relationship, when video data needs to be transmitted between the VPU and the GPU, it is no longer necessary to transfer through the CPU in the host. Instead, direct transmission of data between the VPU and the GPU can be achieved, which can effectively reduce the CPU workload and effectively improve the data transmission efficiency between the VPU and the GPU.

[0075] In one or more embodiments of the present application, the VPU is also used to update the open flag O_RDWR in the firmware, and the first corresponding relationship between the flag and the designated memory.

[0076] Specifically,

[0077] The VPU is also used to update a specific open flag O_RDWR in the firmware, which is used to indicate whether the specified memory area of ​​the VPU is allowed to perform read and write operations. At the same time, the firmware also maintains a first correspondence between the flag and the specified memory, which ensures that when the host or other device accesses the VPU through the flag, it can accurately locate the corresponding specified memory area.

[0078] Specifically, when the VPU receives a command from the host to set or modify access permissions for a certain memory region, the VPU will update the corresponding open flag O_RDWR in its firmware. The state of this flag (such as set or clear) will determine whether subsequent access to the memory region allows read and write operations. In addition, the firmware also maintains the correspondence between the flag and the specified memory through an internal data structure or mapping table so that access requests can be quickly found and verified when needed.

[0079] It should be noted that this correspondence is part of the internal implementation of the VPU firmware and is usually invisible to the host or other external devices. When the host interacts with the VPU through the driver, it follows the interface and protocol provided by the VPU to send commands and receive responses without directly understanding the implementation details inside the firmware. In summary, the VPU updates the open flag O_RDWR in the firmware and maintains the correspondence between the flag and the specified memory, which is one of the key steps to implement VPU memory access control and management. These operations ensure that the VPU can safely and efficiently handle memory access requests from the host or other devices.

[0080] In one or more embodiments of the present application, the host is used to determine the starting address and storage space size corresponding to the first sub-memory by driving the PCIe BAR configured for the VPU; and is also used to determine the starting address and storage space size corresponding to the first sub-memory by driving the PCIe BAR configured for the GPU.

[0081] In actual applications, the host uses the installed driver to configure the PCIe base address register (BAR) of the VPU (video processing unit) to determine the starting physical address corresponding to the first sub-memory area allocated to the VPU and the storage space size of the memory area. Similarly, the host also uses the corresponding driver to configure the PCIe BAR of the GPU (graphics processing unit) to set the starting physical address and the corresponding storage space size for the first sub-memory area allocated to the GPU.

[0082] During the configuration process, the driver interacts with the operating system and hardware devices to ensure that the memory areas allocated to the VPU and GPU do not conflict and can meet their respective work requirements. By accurately configuring the PCIe BAR, the host can effectively manage the VPU and GPU's access to system memory resources, thereby achieving efficient data processing and graphics rendering. It should be noted that although the PCIe BAR configuration process for VPU and GPU is similar, the data types and application scenarios they process are usually different. VPU focuses more on tasks such as video encoding and decoding, image processing, etc., while GPU is mainly used in graphics rendering, parallel computing and other fields. Therefore, when configuring their PCIe BARs, it is necessary to set the appropriate memory area and storage space size according to specific application requirements.

[0083] In one or more embodiments of the present application, the first sub-memory and the second sub-memory store video data of a specified type, including: one or more of YUV raw video data and RGB raw video data.

[0084] In practical applications, the first sub-memory and the second sub-memory are specifically used to store specific types of video data, which include one or a combination of YUV raw video data and RGB raw video data. Video data in YUV format is usually used for video compression and transmission, because it allows independent processing of brightness (Y) and chrominance (U, V) information, thereby achieving more effective data compression. Video data in RGB format directly corresponds to the three primary colors of red (R), green (G), and blue (B) that can be displayed by the display, and is usually used for uncompressed video processing or display scenes.

[0085] In actual applications, the first sub-memory may be allocated to the VPU to store and process YUV raw video data to support efficient video encoding and decoding operations. The second sub-memory may be allocated to the GPU to store and process RGB raw video data for graphics rendering or video post-processing. Of course, depending on the specific application requirements and hardware configuration, these two types of video data can also be stored in the same sub-memory, or transferred and converted between the two sub-memories as needed. In addition, the specific size, storage structure, and access method of the first sub-memory and the second sub-memory will be configured and managed by the host through the driver to ensure the correct storage and efficient processing of video data.

[0086] In one or more embodiments of the present application, when the host sends a transmission request for the specified video data to the VPU, the host obtains the memory address of the first sub-memory from the VPU; when the host sends a transmission request for the specified video data to the GPU, the host obtains the memory address of the second sub-memory from the GPU; the host generates data transmission information based on the memory address of the first sub-memory and the memory address of the second sub-memory; and the PCIe transfers the specified video data in the first sub-memory to the second sub-memory based on the data transmission information.

[0087] When the host needs to send a transfer request for the specified video data to the VPU, it first communicates with the VPU to obtain the specific memory address of the first sub-memory storing the specified video data from the VPU. Similarly, when the host needs to send a transfer request for the same specified video data to the GPU, it communicates with the GPU to obtain the memory address of the second sub-memory storing the video data from the GPU.

[0088] After successfully acquiring the two memory addresses, the host generates detailed data transfer instructions or data transfer information based on this information. This information contains the source address (i.e., the address of the first sub-memory), the target address (i.e., the address of the second sub-memory), and other transmission parameters that may be required, such as the amount of data transmitted, the transmission speed, etc. The host then submits this data transfer information to the PCIe bus system. The PCIe bus will parse this transmission information and efficiently transfer the specified video data stored in the first sub-memory to the second sub-memory according to the instructions therein. In this process, the PCIe bus uses its high-speed serial communication capabilities and advanced error detection and correction mechanisms to ensure the accuracy and reliability of data transmission. Through this series of steps, the host can flexibly schedule and manage video data transmission between the VPU and the GPU to meet complex and changing video processing requirements.

[0089] like Figure 2 A schematic diagram of a video data transmission method provided in an embodiment of the present application. The method is applied to the VPU end. Figure 2 As can be seen in , the method includes the following steps:

[0090] Step 201: In response to the host sending a request for the specified video data transmission, a pre-configured memory address of a first sub-memory is sent to the host; so that the host uses the memory address of the first sub-memory and the memory address of the second sub-memory of the GPU to send data transmission information to PCIe.

[0091] Step 202: In response to the data transmission instruction sent by the PCIe, send the specified video data in the first sub-memory to the second sub-memory of the GPU.

[0092] Among them, before sending the memory address of the pre-configured first sub-memory to the host in step 201, it also includes: setting configuration parameters of the first mapping relationship between the specified memory in the VPU and the PCIe memory space; selecting part of the first sub-memory from the specified memory as P2P memory according to the first mapping relationship, so as to establish a second mapping relationship between the first sub-memory and the host memory.

[0093] Before sending the memory address of the pre-configured first sub-memory to the host, the VPU first sets configuration parameters of the first mapping relationship between its internal designated memory and the PCIe memory space. These parameters define how the VPU memory is mapped to the PCIe address space so that the host and other PCIe devices can access it.

[0094] Next, the VPU selects a portion of its designated memory as the P2P memory, namely the first sub-memory, according to the set first mapping relationship. This portion of memory is specifically used for high-speed data transmission with the GPU or other PCIe devices.

[0095] The VPU is also responsible for establishing a second mapping relationship between the first sub-memory and the host memory, which is usually implemented through an operating system and a driver to ensure that the host can directly access the first sub-memory.

[0096] When the host sends a request for the specified video data transmission, the VPU responds to the request and sends the pre-configured memory address of the first sub-memory to the host. In this way, the host can use this address and the memory address of the second sub-memory previously obtained from the GPU to send data transmission information to PCIe.

[0097] After receiving the data transmission instruction sent by PCIe according to the data transmission information of the host, the VPU responds to this instruction and sends the specified video data stored in its first sub-memory to the second sub-memory of the GPU. This transmission process is realized through the PCIe bus, which provides a high-speed, low-latency data channel.

[0098] Through these two steps, the VPU can work with the GPU to achieve efficient transmission and processing of specified video data. At the same time, the configuration of the VPU and the setting of the mapping relationship ensure the accuracy and reliability of data transmission.

[0099] like Figure 3 A schematic diagram of another method for transmitting video data provided in an embodiment of the present application. The method is applied to a GPU. Figure 3As shown, the method specifically comprises the following steps:

[0100] Step 301: In response to the host sending a request for transmitting the specified video data, the memory address of the pre-configured second sub-memory is sent to the host. Step 302: The PCIe transfers the specified video data in the first sub-memory to the second sub-memory according to the data transmission information. Before sending the pre-configured memory address of the second sub-memory to the host in step 301, it also includes: setting configuration parameters of a third mapping relationship between the specified memory in the GPU and the PCIe memory space; selecting part of the second sub-memory from the specified memory as the P2P memory according to the third mapping relationship, so as to establish a fourth mapping relationship between the second sub-memory and the host memory.

[0101] Before sending the memory address of the pre-configured second sub-memory to the host, the GPU first sets the configuration parameters of the third mapping relationship between its internal designated memory and the PCIe memory space. These parameters define in detail how the GPU memory is mapped to the address space of PCIe so that the host and other PCIe devices (such as VPU) can efficiently access the GPU memory. Then, the GPU selects a part of its designated memory as P2P memory, that is, the second sub-memory, based on the set third mapping relationship. This part of the memory is specially divided out for high-speed data transmission and sharing with the host, VPU or other PCIe devices. The GPU is also responsible for establishing a fourth mapping relationship between the second sub-memory and the host memory. This is usually achieved through the operating system, driver and GPU hardware support to ensure that the host can directly and efficiently access the second sub-memory.

[0102] When the host sends a request for the specified video data transmission, the GPU responds to the request and sends the pre-configured memory address of the second sub-memory to the host. In this way, the host can use this address and the address of the first sub-memory that may have been obtained from the VPU before (if data transmission from the VPU to the GPU is involved) to construct data transmission information and perform data transmission through PCIe.

[0103] After receiving the data transfer instruction sent by PCIe according to the data transfer information of the host, the PCIe bus is responsible for transferring the specified video data in the first sub-memory of the VPU (or directly sent by the host if the data transfer does not involve the VPU) to the second sub-memory of the GPU. This transmission process fully utilizes the high-speed and low-latency characteristics of the PCIe bus to ensure the efficiency and accuracy of data transmission.

[0104] Through these two steps, the GPU can work with the host and VPU (if involved) to achieve efficient transmission and processing of specified video data in the system. At the same time, the GPU mapping relationship setting and P2P memory selection provide a solid foundation and guarantee for data transmission.

[0105] like Figure 4 A flowchart of another video data transmission method provided in an embodiment of the present application is shown in FIG. The method is applied to a host. Figure 4 As shown, the method specifically comprises the following steps:

[0106] Step 401: Select a portion of the first sub-memory from the VPU designated memory as the P2P memory, and establish a second mapping relationship between the first sub-memory and the host memory; wherein the VPU designated memory is the first mapping relationship between the VPU and the PCIe memory space configured by setting configuration parameters.

[0107] Step 402: Select a portion of the second sub-memory from the designated memory of the GPU as the P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory; wherein the designated memory of the GPU is the third mapping relationship between the GPU and the PCIe memory space by setting configuration parameters.

[0108] Step 403: Implement designated video data transmission between the first sub-memory and the second sub-memory according to the third mapping relationship or the fourth mapping relationship.

[0109] Figure 5 1 is a block diagram of a video data transmission device according to an exemplary embodiment. The device is applied to a VPU and specifically includes:

[0110] The first sending module 51 is used to send a pre-configured memory address of a first sub-memory to the host in response to the host sending a request for the specified video data transmission; so that the host uses the memory address of the first sub-memory and the memory address of the second sub-memory of the GPU to send data transmission information to PCIe.

[0111] The second sending module 52 sends the specified video data in the first sub-memory to the second sub-memory of the GPU in response to the data transmission instruction sent by the PCIe.

[0112] Figure 6 1 is a block diagram of another video data transmission device according to an exemplary embodiment. The device is applied to a GPU and specifically includes:

[0113] The first sending module 61 is configured to send a memory address of a pre-configured second sub-memory to the host in response to the host sending a transmission request for the designated video data.

[0114] The second sending module 62 is used for the PCIe to transfer the specified video data in the first sub-memory to the second sub-memory according to the data transmission information.

[0115] Figure 7 The block diagram of a video data transmission device according to an exemplary embodiment is shown. The device is applied to a host, and the device includes:

[0116] The first relationship establishing module 71 is used to select part of the first sub-memory from the VPU designated memory as P2P memory, and establish a second mapping relationship between the first sub-memory and the host memory; wherein the VPU designated memory is the first mapping relationship between the VPU and the PCIe memory space configured by setting configuration parameters.

[0117] The second relationship establishing module 72 is used to select a part of the second sub-memory from the designated memory of the GPU as the P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory; wherein the designated memory of the GPU is the third mapping relationship between the GPU and the PCIe memory space by setting configuration parameters.

[0118] The data transmission module 73 is used to implement the specified video data transmission between the first sub-memory and the second sub-memory according to the third mapping relationship or the fourth mapping relationship.

[0119] An embodiment of the present application further proposes an electronic device, comprising a processor and a memory; the memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the video data transmission method of any of the above embodiments.

[0120] The embodiments of the present application further provide a computer-readable storage medium. When an executable computer program in the storage medium is executed by a processor, the video data transmission method of any of the above embodiments can be implemented.

[0121] Regarding the device in the above embodiment, the specific manner in which the processor performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0122] Figure 8 1 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 1200 may be provided as a server. Figure 8, the device 1200 includes a processing component 1222, which further includes one or more processors, and a memory resource represented by a memory 1232 for storing instructions, such as applications, that can be executed by the processing component 1222. The application stored in the memory 1232 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1222 is configured to execute instructions to perform the above-mentioned video data transmission method.

[0123] The device 1200 may also include a power component 1226 configured to perform power management of the device 1200, a wired or wireless network interface 1250 configured to connect the device 1200 to a network, and an input / output (I / O) interface 1258. The device 1200 may operate based on an operating system stored in the memory 1232, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.

[0124] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1232 including instructions, which can be executed by the processing component 1222 of the device 1200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0125] In the present invention, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. The term "plurality" refers to two or more than two, unless otherwise clearly defined.

[0126] The above description of the embodiments is to facilitate those of ordinary skill in the art to understand and apply the present application. It is obvious that those familiar with the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without having to pay creative labor. Therefore, the present application is not limited to the embodiments herein, and improvements and modifications made by those skilled in the art based on the contents disclosed in the present application without departing from the scope and spirit of the present application are within the scope of the present application.

Claims

1. A video data transmission system, characterized in that: The system includes: a VPU, a GPU and a host. The VPU is used to set configuration parameters of the VPU, and the configuration parameters are used to configure a first mapping relationship between a specified memory in the VPU and a PCIe memory space; The host is used to select a part of the first sub-memory from the designated memory of the VPU as the P2P memory, and establish a second mapping relationship between the first sub-memory and the host memory; The GPU is used to set configuration parameters of the GPU, and the configuration parameters are used to configure a third mapping relationship between a specified memory in the GPU and a PCIe memory space; The host is used to select a part of the second sub-memory from the designated memory of the GPU as the P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory; The host is used to implement designated video data transmission between the first sub-memory and the second sub-memory according to the third mapping relationship or the fourth mapping relationship.

2. The system according to claim 1, characterized in that The VPU is also used to open the flag bit O_RDWR in the update firmware, and the first corresponding relationship between the flag bit and the designated memory.

3. The system according to claim 1, characterized in that The host is used to determine the starting address and storage space size corresponding to the first sub-memory by driving the PCIeBAR configured by the VPU; It is also used to determine the starting address and storage space size corresponding to the first sub-memory by driving the PCIe BAR configured for the GPU.

4. The system according to claim 1, characterized in that The first sub-memory and the second sub-memory store video data of a specified type, including one or more of YUV raw video data and RGB raw video data.

5. The system according to claim 1, characterized in that When the host sends a data transmission request for the designated video to the VPU, the host obtains a memory address of the first sub-memory from the VPU; When the host sends a data transmission request for the designated video to the GPU, the host obtains a memory address of the second sub-memory from the GPU; The host generates data transmission information according to the memory address of the first sub-memory and the memory address of the second sub-memory; The PCIe transfers the designated video data in the first sub-memory to the second sub-memory according to the data transmission information.

6. A video data transmission method, characterized in that: Applied to a VPU, the method comprises: In response to a host sending a data transmission request for a specified video, a pre-configured memory address of a first sub-memory is sent to the host, so that the host uses the memory address of the first sub-memory and the memory address of the second sub-memory of the GPU to send data transmission information to PCIe; In response to the data transmission instruction sent by the PCIe, the specified video data in the first sub-memory is sent to the second sub-memory of the GPU.

7. The method according to claim 6, characterized in that Before sending the pre-configured memory address of the first sub-memory to the host, the method further includes: Setting configuration parameters of a first mapping relationship between a specified memory in the VPU and a PCIe memory space; A portion of the first sub-memory is selected from the designated memory as a P2P memory according to the first mapping relationship, so as to establish a second mapping relationship between the first sub-memory and the host memory.

8. A video data transmission method, characterized in that: Applied to a GPU, the method comprises: In response to a host sending a data transmission request for a specified video, sending a memory address of a preconfigured second sub-memory to the host; PCIe transfers the designated video data in the first sub-memory to the second sub-memory according to the data transmission information.

9. The method according to claim 8, characterized in that Before sending the pre-configured memory address of the second sub-memory to the host, the method further includes: Set the configuration parameters of the third mapping relationship between the specified memory in the GPU and the PCIe memory space; A portion of the second sub-memory is selected from the designated memory as the P2P memory according to the third mapping relationship, so as to establish a fourth mapping relationship between the second sub-memory and the host memory.

10. A video data transmission method, characterized in that: Applied to a host, the method comprises: Selecting a portion of the first sub-memory from the VPU designated memory as the P2P memory, and establishing a second mapping relationship between the first sub-memory and the host memory; wherein the VPU designated memory is a first mapping relationship between the VPU and the PCIe memory space configured by setting configuration parameters; Select a portion of the second sub-memory from the GPU designated memory as the P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory; wherein the GPU designated memory is the third mapping relationship between the GPU and the PCIe memory space by setting configuration parameters; The designated video data transmission between the first sub-memory and the second sub-memory is implemented according to the third mapping relationship or the fourth mapping relationship.

11. A video data transmission device, characterized in that: Applied to VPU, the device comprises: A first sending module, configured to send a pre-configured memory address of a first sub-memory to the host in response to a specified video data transmission request sent by the host, so that the host uses the memory address of the first sub-memory and the memory address of the second sub-memory of the GPU to send data transmission information to the PCIe; The second sending module sends the specified video data in the first sub-memory to the second sub-memory of the GPU in response to the data transmission instruction sent by the PCIe.

12. A video data transmission device, characterized in that: Applied to a GPU, the device comprises: A first sending module, configured to send a memory address of a pre-configured second sub-memory to the host in response to a request for transmitting a specified video data from the host; The second sending module is used for PCIe to transfer the specified video data in the first sub-memory to the second sub-memory according to the data transmission information.

13. A video data transmission device, characterized in that: Applied to a host, the device comprises: A first relationship establishment module is used to select a portion of the first sub-memory from the VPU designated memory as the P2P memory, and establish a second mapping relationship between the first sub-memory and the host memory; wherein the VPU designated memory is a first mapping relationship between the VPU and the PCIe memory space configured by setting configuration parameters; A second relationship establishment module is used to select a part of the second sub-memory from the GPU designated memory as the P2P memory, and establish a fourth mapping relationship between the second sub-memory and the host memory; wherein the GPU designated memory is the third mapping relationship between the GPU and the PCIe memory space by setting configuration parameters; A data transmission module is used to implement the specified video data transmission between the first sub-memory and the second sub-memory according to the third mapping relationship or the fourth mapping relationship.

14. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method according to any one of claims 6 to 10.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the executable computer program in the storage medium is executed by a processor, the method according to any one of claims 6 to 10 can be implemented.

Images