Image transmission operation acceleration method and device based on GPGPU and storage medium

By building a video input control unit and interrupt signal mechanism in the GPGPU, the data of the image acquisition device is directly transmitted to the GPGPU, which solves the problems of slow image processing speed and poor real-time performance, and achieves more efficient data processing.

CN119071550APending Publication Date: 2024-12-03SHANDONG INSPUR SCI RES INST CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411097802.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the fields of image processing and computer vision, external data is collected and stored on the host side and transmitted to the GPGPU, resulting in slower processing of video or image data, affecting real-time.

Method used

A GPGPU-based image transmission operation acceleration method is designed. By constructing a video input control unit, the data of the image acquisition device is directly transmitted to the GPGPU to avoid the data transfer process on the host side, and the GPGPU is notified to perform operations using the interrupt signal mechanism.

Benefits of technology

It improves data processing speed and real-time performance, directly sends data from the image acquisition device to the GPGPU, reducing the host data transmission process and improving the efficiency of image processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071550A_ABST
    Figure CN119071550A_ABST
Patent Text Reader

Abstract

An image transmission operation acceleration method and device based on a GPGPU and a storage medium relate to the field of GPGPUs, external image acquisition devices are closely combined, and a video transmission interface is integrated on the GPGPU, so that data acquired by the image acquisition devices can be directly sent to the GPGPU, the process of moving the data from a host can be avoided, and the data transmission speed is improved. And the speed and the real-time performance of data processing are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of GPGPU, and in particular to a method, device and storage medium for accelerating image transmission operations based on GPGPU. Background Art

[0002] With the use of efficient computing tools such as GPGPU, artificial intelligence has received widespread attention and has been applied in many scenarios. Neural networks have made great progress, presenting a huge demand for computing power. Breakthroughs have been made in the fields of image processing and computer vision. With the deepening of research and the development of technology, the application of neural networks in the field of image processing will become more and more extensive.

[0003] It has been widely developed in image recognition, face detection, target detection, vehicle and traffic monitoring, autonomous driving, etc. GPGPU greatly enhances the computing speed of neural networks due to its parallel computing.

[0004] The collection of external data is usually completed by specialized hardware devices, such as cameras, microphones, sensors, etc. These devices are responsible for capturing data from the physical world and converting it into digital signals. In some cases, the collected data may first be stored in the host's memory or disk, and then sent to the GPGPU by the host. After the GPGPU is processed, it is returned to the host. This often affects the processing speed of video or image data and the real-time performance of image processing. Summary of the invention

[0005] In order to overcome the deficiencies of the above technologies, the present invention provides a method, device and storage medium for accelerating image transmission operations based on GPGPU to increase data processing speed and real-time performance.

[0006] The technical solution adopted by the present invention to overcome the technical problems is: A method for accelerating image transmission operations based on GPGPU, comprising: S1. Construct a video input control unit; S2. The image acquisition device transmits the acquired image data to the video input control unit, which converts the image into an RGB format image; S3. When the cache unit of the GPGPU receives N RGB format images, an interrupt source signal is generated and the interrupt source signal is transmitted to the interrupt processing unit of the GPGPU; S4. After receiving the interrupt source signal, the GPGPU interrupt processing unit informs the host; S5. After receiving the interrupt source signal, the host determines the interrupt type and issues a control instruction to inform the GPGPU SM core to perform the calculation operation; S6. After the SM core completes the operation, it returns the operation result to the host.

[0007] Further, step S1 includes the following steps: S1-1. The video input control unit is composed of an interface protocol circuit that matches the interface protocol of the image acquisition device, an image format conversion module, a feature value cache, and a NOC interface; S1-2. The video input control unit communicates with the image acquisition device via an interface protocol circuit; S1-3. The interface protocol circuit transfers the captured image to the image format conversion module, converts it into RGB format, and then stores the image in RGB format into the feature value cache; S1-4. The video input control unit is connected to the NOC on-chip network through the NOC interface.

[0008] Furthermore, in step S2, the image acquisition device is a camera.

[0009] Furthermore, the Host CPU communicates with the GPGPU via a PCIE interface.

[0010] Furthermore, the video input control unit, the cache unit of the GPGPU, and the SM core of the GPGPU are all mounted on the NOC on-chip network.

[0011] Furthermore, the above-mentioned host is Host CPU.

[0012] Furthermore, the video input control unit is node 1 of the NOC on-chip network, the cache unit of the GPGPU is node 2 and node 4 of the NOC on-chip network, and the SM core of the GPGPU is node 3 of the NOC on-chip network. After receiving the control instruction, the SM core goes to node 1, node 2, and node 4 through the NOC on-chip network to index and calculate the eigenvalue data.

[0013] On the other hand, the present invention also relates to a device for accelerating image transmission operations based on GPGPU, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: The memory is used to store computer programs; The processor is configured to execute, by running a program stored in the memory: The image acquisition device transmits the acquired image data to the video input control unit, and the video input control unit converts the image into an RGB format image; When the cache unit of the GPGPU receives N RGB format images, an interrupt source signal is generated, and the interrupt source signal is transmitted to the interrupt processing unit of the GPGPU; After receiving the interrupt source signal, the interrupt processing unit of the GPGPU notifies the host; After receiving the interrupt source signal, the host determines the interrupt type and issues a control instruction to inform the GPGPU SM core to perform the calculation operation; After the SM core completes the operation, it returns the operation result to the host.

[0014] On the other hand, the present invention also relates to a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements: The image acquisition device transmits the acquired image data to the video input control unit, and the video input control unit converts the image into an RGB format image; When the cache unit of the GPGPU receives N RGB format images, an interrupt source signal is generated, and the interrupt source signal is transmitted to the interrupt processing unit of the GPGPU; After receiving the interrupt source signal, the interrupt processing unit of the GPGPU notifies the host; After receiving the interrupt source signal, the host determines the interrupt type and issues a control instruction to inform the GPGPU SM core to perform the calculation operation; After the SM core completes the operation, it returns the operation result to the host.

[0015] The beneficial effects of the present invention are: closely combining an external image acquisition device and integrating a video transmission interface on a GPGPU, so that the data collected by the image acquisition device can be directly sent to the GPGPU, thus avoiding the process of moving data from the host and increasing the speed and real-time performance of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a structural schematic diagram of a GPGPU with integrated video interface of the present invention; Figure 2 is a structural diagram of a video input control unit of the present invention; Figure 3 It is a schematic diagram of a NOC node; Figure 4 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0017] The following is attached Figure 1 To Attachment Figure 4 The preferred embodiments of the present invention are described in detail so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0018] In the description of the present invention, it should be noted that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0019] The terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0020] Example 1 As attached Figure 4 As shown, a method for accelerating image transmission operations based on GPGPU includes: S1. Build a video input control unit.

[0021] S2. The image acquisition device transmits the acquired image data to the video input control unit, and the video input control unit converts the image into an RGB format image.

[0022] S3. After the cache unit of the GPGPU receives N RGB format images, an interrupt source signal is generated and the interrupt source signal is transmitted to the interrupt processing unit of the GPGPU.

[0023] S4. After receiving the interrupt source signal, the interrupt processing unit of GPGPU informs the host.

[0024] S5. After receiving the interrupt source signal, the host determines the interrupt type and issues a control instruction to inform the SM core of the GPGPU to perform the operation. The SM core is the stream multiprocessor of the GPGPU, which is equivalent to a processor core.

[0025] S6. After the SM core completes the operation, it returns the operation result to the host.

[0026] As attached Figure 1As shown, the image acquisition device collects image data and transmits it to the video input control unit. After the video input control unit processes the image data, it notifies the interrupt processing unit by sending an interrupt signal, and the interrupt processing unit further notifies the Host CPU to perform the next step. The Host CPU communicates with the GPGPU through the PCIE high-speed interconnect interface unit. After that, the Host CPU controls the SM core to fetch data from the cache unit or the video input control unit to perform data operations, wherein the video input control unit, the cache unit, and the SM core are all mounted on the NoC network, and they communicate with each other through the NoC network.

[0027] In this embodiment, as shown in the attached Figure 2 As shown, step S1 includes the following steps: S1-1. The video input control unit is composed of an interface protocol circuit that matches the interface protocol of the image acquisition device, an image format conversion module, a feature value cache, and a NOC interface.

[0028] S1-2. The video input control unit is connected to the image acquisition device through an interface protocol circuit.

[0029] S1-3. The interface protocol circuit transmits the collected image to the image format conversion module, converts it into RGB format, and then stores the RGB format image in the feature value cache.

[0030] S1-4. The video input control unit is connected to the NOC on-chip network through the NOC interface.

[0031] In this embodiment, the image acquisition device in step S2 is a camera.

[0032] In this embodiment, the Host CPU communicates with the GPGPU via a PCIE interface.

[0033] In this embodiment, the video input control unit, the cache unit of the GPGPU, and the SM core of the GPGPU are all mounted on the NOC on-chip network.

[0034] In this embodiment, the host is Host CPU.

[0035] In this embodiment, as shown in the attached Figure 3 As shown, the video input control unit is node 1 of the NOC on-chip network, the cache unit of the GPGPU is node 2 and node 4 of the NOC on-chip network, and the SM core of the GPGPU is node 3 of the NOC on-chip network. After receiving the control instruction, the SM core goes to node 1, tangent point 2, and node 4 through the NOC on-chip network to index and calculate the eigenvalue data.

[0036] Example 2 A device for accelerating image transmission operations based on GPGPU, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: The memory is used to store computer programs.

[0037] The processor is configured to execute, by running a program stored in the memory: The image acquisition device transmits the acquired image data to the video input control unit, and the video input control unit converts the image into an RGB format image.

[0038] When the cache unit of the GPGPU receives N RGB format images, an interrupt source signal is generated and the interrupt source signal is transmitted to the interrupt processing unit of the GPGPU.

[0039] After receiving the interrupt source signal, the interrupt processing unit of the GPGPU informs the host.

[0040] After receiving the interrupt source signal, the host determines the interrupt type and issues a control instruction to inform the GPGPU SM core to perform the calculation operation.

[0041] After the SM core completes the operation, it returns the operation result to the host.

[0042] Example 3 A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements: The image acquisition device transmits the acquired image data to the video input control unit, and the video input control unit converts the image into an RGB format image.

[0043] When the cache unit of the GPGPU receives N RGB format images, an interrupt source signal is generated and the interrupt source signal is transmitted to the interrupt processing unit of the GPGPU.

[0044] After receiving the interrupt source signal, the interrupt processing unit of the GPGPU informs the host.

[0045] After receiving the interrupt source signal, the host determines the interrupt type and issues a control instruction to inform the GPGPU SM core to perform the calculation operation.

[0046] After the SM core completes the operation, it returns the operation result to the host.

[0047] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0048] It should also be understood that in the embodiments of this document, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this document generally indicates that the associated objects before and after are in an "or" relationship.

[0049] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this article.

[0050] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0051] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0052] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.

[0053] In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.

[0054] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0055] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for accelerating image transmission operations based on GPGPU. It is characterized in that include: S1. Construct a video input control unit; S2. The image acquisition device transmits the acquired image data to the video input control unit, which converts the image into an RGB format image; S3. When the cache unit of the GPGPU receives N RGB format images, an interrupt source signal is generated and the interrupt source signal is transmitted to the interrupt processing unit of the GPGPU; S4. After receiving the interrupt source signal, the GPGPU interrupt processing unit informs the host; S5. After receiving the interrupt source signal, the host determines the interrupt type and issues a control instruction to inform the GPGPU SM core to perform the calculation operation; S6. After the SM core completes the operation, it returns the operation result to the host.

2. The method for accelerating image transmission operation based on GPGPU according to claim 1, It is characterized in that Step S1 includes the following steps: S1-1. The video input control unit is composed of an interface protocol circuit that matches the interface protocol of the image acquisition device, an image format conversion module, a feature value cache, and a NOC interface; S1-2. The video input control unit communicates with the image acquisition device via an interface protocol circuit; S1-3. The interface protocol circuit transfers the captured image to the image format conversion module, converts it into RGB format, and then stores the image in RGB format into the feature value cache; S1-4. The video input control unit is connected to the NOC on-chip network through the NOC interface.

3. The method for accelerating image transmission operation based on GPGPU according to claim 1, Features: In step S2, the image acquisition device is a camera.

4. The method for accelerating image transmission operation based on GPGPU according to claim 1, Features: HostCPU and GPGPU communicate through the PCIE interface.

5. The method for accelerating image transmission operation based on GPGPU according to claim 1, Features: The video input control unit, GPGPU cache unit, and GPGPU SM core are all mounted on the NOC on-chip network.

6. The method for accelerating image transmission operation based on GPGPU according to claim 1, Features: The host is Host CPU.

7. The method for accelerating image transmission operation based on GPGPU according to claim 5, Features: The video input control unit is node 1 of the NOC on-chip network, the cache unit of the GPGPU is node 2 and node 4 of the NOC on-chip network, and the SM core of the GPGPU is node 3 of the NOC on-chip network. After receiving the control instruction, the SM core goes to node 1, tangent point 2, and node 4 through the NOC on-chip network to index and calculate the eigenvalue data.

8. A device for accelerating image transmission operations based on GPGPU. It is characterized in that include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: The memory is used to store computer programs; The processor is configured to execute, by running a program stored in the memory: The image acquisition device transmits the acquired image data to the video input control unit, and the video input control unit converts the image into an RGB format image; When the cache unit of the GPGPU receives N RGB format images, an interrupt source signal is generated, and the interrupt source signal is transmitted to the interrupt processing unit of the GPGPU; After receiving the interrupt source signal, the interrupt processing unit of the GPGPU notifies the host; After receiving the interrupt source signal, the host determines the interrupt type and issues a control instruction to inform the GPGPU SM core to perform the calculation operation; After the SM core completes the operation, it returns the operation result to the host.

9. A computer-readable storage medium, It is characterized in that The computer readable storage medium stores a computer program, which, when executed by a processor, implements: The image acquisition device transmits the acquired image data to the video input control unit, and the video input control unit converts the image into an RGB format image; When the cache unit of the GPGPU receives N RGB format images, an interrupt source signal is generated, and the interrupt source signal is transmitted to the interrupt processing unit of the GPGPU; After receiving the interrupt source signal, the interrupt processing unit of the GPGPU notifies the host; After receiving the interrupt source signal, the host determines the interrupt type and issues a control instruction to inform the GPGPU SM core to perform the calculation operation; After the SM core completes the operation, it returns the operation result to the host.