Image processing apparatus
By introducing GPU and CPU units into LED image processing equipment, combined with FPGA units, and using PCIe and network switches for data transmission, the problems of long development cycles and unclear image quality caused by FPGA cascading are solved, achieving efficient and fast image processing and high-quality display.
Patent Information
- Application Number
- CN202423321494.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2026-01-27
- Estimated Expiration
- 2034-12-31
AI Technical Summary
Existing LED image processing solutions rely on FPGA cascading, resulting in long development cycles, high difficulty, limited resources, and insufficient image quality, making it difficult to adapt to highly complex algorithms and new requirements.
Image processing is performed using a graphics processing unit (GPU), combined with a central processing unit (CPU) and a field-programmable gate array (FPGA). Data transmission is handled through a PCIe switch and a network switch. This system constructs an image processing device that leverages the parallel processing capabilities of the GPU and machine learning algorithms to improve image processing accuracy.
It shortened the development cycle, reduced the development difficulty, improved the speed and efficiency of image processing, enhanced the display quality, and adapted to complex image processing needs.
Smart Images
Figure CN223842604U_ABST
Abstract
Description
Technical Field
[0001] This utility model relates to the field of LED technology, and in particular to an image processing device. Background Technology
[0002] Currently, image processing solutions in the LED (Light Emitting Diode) industry primarily rely on FPGA (Field-Programmable Gate Array) designs. For example, multiple FPGAs are cascaded to process images in a video stream. However, with the development of LED technology and the increasing demands of the market for image processing, using cascaded FPGA architectures for heterogeneous image processing results in images with insufficient clarity. Furthermore, FPGAs for image processing in the LED field suffer from problems such as long development cycles, high difficulty, and limited resources. Utility Model Content
[0003] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this utility model provides an image processing device that solves the problems of long development cycle, difficulty in implementing highly complex algorithms, and difficulty in adapting to new requirements that only use FPGA cascade architecture.
[0004] To achieve the above objectives, the present invention provides the following technical solution:
[0005] An embodiment of this utility model provides an image processing device. The first image processing device includes: an input unit, a video processing unit, an output unit, and an LED display screen. The input unit is connected to the video processing unit, the video processing unit is connected to the output unit, and the output unit is connected to the LED display screen.
[0006] The input unit is used to acquire a video stream, decode the video stream to obtain initial image data, and send the initial image data to the video processing unit;
[0007] The video processing unit is used to process the initial image data to obtain intermediate image data, and send the intermediate image data to the output unit;
[0008] The video processing unit includes a graphics processing unit (GPU); specifically, the video processing unit is used to process the initial image data through the GPU to obtain intermediate image data.
[0009] The output unit is used to process the intermediate image data to obtain display data, and then send the display data to the LED display screen for display.
[0010] As an optional embodiment of this utility model, the video processing unit further includes a central processing unit (CPU) unit, which is used to process the initial image data through the CPU unit to obtain intermediate image data.
[0011] As an optional embodiment of this utility model, the video processing unit is further configured to process the initial image data through the GPU unit and the CPU unit to obtain intermediate image data.
[0012] As an optional embodiment of this utility model, the video processing unit further includes a field-programmable gate array (FPGA) unit, which is used to process the initial image data through the FPGA unit to obtain intermediate image data.
[0013] As an optional embodiment of this utility model, the video processing unit is further configured to process the initial image data through the GPU unit, the CPU unit, and the FPGA unit to obtain intermediate image data.
[0014] As an optional embodiment of this utility model, the video processing unit further includes a PCIe switch, and the CPU unit is connected to the GPU unit and the FPGA unit through the PCIe switch.
[0015] As an optional embodiment of this utility model, the PCIe switch includes a PCIe interface, and the CPU unit is connected to the GPU unit and the FPGA unit through the PCIe interface.
[0016] As an optional embodiment of this utility model, the device further includes a network switch, which is used to transmit the video stream to at least one second image processing device.
[0017] As an optional embodiment of this utility model, the network switch is further configured to transmit the initial image data to at least one second image processing device.
[0018] As an optional embodiment of this utility model, the network switch is further configured to transmit the intermediate image data to at least one second image processing device.
[0019] As an optional embodiment of this utility model, the network switch is further configured to transmit the display data to at least one second image processing device.
[0020] As an optional embodiment of this utility model, the LED display screen includes at least one cabinet, and each cabinet includes at least one receiving card, which is used to drive the LED display screen to light up the LED display screen.
[0021] As an optional embodiment of this utility model, the device includes at least one GPU unit.
[0022] The image processing device provided by this utility model includes an input unit, a video processing unit, and an output unit. The input unit is connected to the video processing unit, the video processing unit is connected to the output unit, and the output unit is connected to an LED display screen. The input unit is used to acquire a video stream, decode the video stream to obtain initial image data, and send the initial image data to the video processing unit. The video processing unit is used to process the initial image data to obtain intermediate image data and send the intermediate image data to the output unit. The video processing unit includes a graphics processing unit (GPU). Specifically, the video processing unit processes the initial image data through the GPU to obtain intermediate image data. The output unit processes the intermediate image data to obtain display data and sends the display data to the LED display screen for display. During image processing, because the GPU unit focuses on parallel processing of image data and can perform complex image processing algorithms, processing the initial image through the GPU unit reduces the workload compared to relying solely on the FPGA for image processing. Furthermore, because the GPU can perform image processing through machine learning algorithms, and the processing algorithm accuracy of the GPU is higher than that of traditional FPGA fixed-point image processing, it can improve the display quality. Attached Figure Description
[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the present invention and, together with the description, serve to explain the principles of the present invention.
[0024] To more clearly illustrate the technical solutions in the embodiments of this utility model or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is one of the structural schematic diagrams of an image processing device in one embodiment;
[0026] Figure 2 This is a second schematic diagram of the structure of an image processing device in one embodiment;
[0027] Figure 3 This is the third schematic diagram of the structure of the image processing device in one embodiment;
[0028] Figure 4 This is a fourth schematic diagram of the structure of an image processing device in one embodiment;
[0029] Figure 5 This is the fifth schematic diagram of the structure of an image processing device in one embodiment;
[0030] Figure 6 This is a schematic diagram of the structure of an image processing device in one embodiment. Detailed Implementation
[0031] To better understand the above-mentioned objectives, features, and advantages of this utility model, the solution of this utility model will be further described below. It should be noted that, unless otherwise specified, the embodiments of this utility model and the features thereof can be combined with each other.
[0032] Many specific details are set forth in the following description in order to provide a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the present invention, and not all embodiments.
[0033] The terms "first" and "second" and other relational terms used in the specification and claims of this utility model are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.
[0034] In the embodiments of this utility model, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this utility model should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. Furthermore, in the description of the embodiments of this utility model, unless otherwise stated, "a plurality of" means two or more.
[0035] In one embodiment, such as Figure 1As shown, a first image processing device 100 is provided, comprising: an input unit 101, a video processing unit 102, and an output unit 103. The input unit 101 is connected to the video processing unit 102, the video processing unit 102 is connected to the output unit 103, and the output unit 103 is connected to an LED display screen 104. The input unit 101 is used to acquire a video stream, decode the video stream to obtain initial image data, and send the initial image data to the video processing unit 102. The video processing unit 102 is used to process the initial image data to obtain intermediate image data and send the intermediate image data to the output unit 103. The video processing unit 102 includes a graphics processing unit (GPU) 1021. Specifically, the video processing unit 102 is used to process the initial image data through the GPU 1021 to obtain intermediate image data. The output unit 103 is used to process the intermediate image data to obtain display data and send the display data to the LED display screen 104 for display.
[0036] The beneficial effects of this utility model are as follows: The first image processing device includes an input unit, a video processing unit, and an output unit. The input unit is connected to the video processing unit, the video processing unit is connected to the output unit, and the output unit is connected to the LED display screen. The input unit is used to acquire a video stream, decode the video stream to obtain initial image data, and send the initial image data to the video processing unit. The video processing unit is used to process the initial image data to obtain intermediate image data and send the intermediate image data to the output unit. The video processing unit includes a graphics processing unit (GPU). Specifically, the video processing unit processes the initial image data through the GPU to obtain intermediate image data. The output unit processes the intermediate image data to obtain display data and sends the display data to the LED display screen for display. During image processing, because the GPU unit focuses on parallel processing of image data and can perform complex image processing algorithms, processing the initial image through the GPU unit reduces the workload compared to relying solely on the FPGA for image processing. Furthermore, because the GPU can perform image processing through machine learning algorithms, and the processing algorithm accuracy of the GPU is higher than that of traditional FPGA fixed-point image processing, it can improve the display quality.
[0037] Specifically, GPUs, with their numerous parallel processing units, are well-suited for handling large-scale parallel computing tasks such as convolutional neural networks (CNNs) and image filtering. Furthermore, GPUs have high-bandwidth video memory, making them suitable for processing large amounts of data. Meanwhile, NVIDIA and AMD (graphics processing and related technology companies) provide abundant development tools and libraries, such as CUDA (Compute Unified Device Architecture, a general-purpose parallel computing platform and programming model), OpenCL (Open Computing Language), and cuDNN (CUDA Deep Neural Network, a GPU-accelerated library for deep neural networks), offering a wealth of open-source libraries and community support, facilitating rapid development. Therefore, by using GPU units to process initial image data, intermediate image data is obtained and sent to the output unit. Utilizing GPUs for image processing can significantly shorten the development cycle, reduce development difficulty, and improve processing speed and efficiency.
[0038] In some embodiments, the video processing unit further includes a central processing unit (CPU) unit, and the video processing unit is further configured to process the initial image data through the CPU unit and the GPU unit to obtain intermediate image data.
[0039] Specifically, in Figure 1 Based on, refer to Figure 2 As shown, the video processing unit 102 also includes a CPU unit 1022. The video processing unit is used to process the initial image data through the GPU unit 1021 and the CPU unit 1022 to obtain intermediate image data. Since the CPU is a general-purpose processor, it is suitable for various computing tasks, including control logic and complex algorithms in image processing. Furthermore, the CPU can be programmed using high-level languages (such as C / C++, Python), resulting in a short development cycle and ease of maintenance; it has extensive development tools and library support, such as OpenCV and NumPy, facilitating quick learning for developers; and it can handle multiple tasks simultaneously, making it suitable for scenarios requiring multi-threading and multi-tasking. Therefore, the initial image data can be processed by the CPU unit to obtain intermediate image data. That is, the CPU also participates in image data processing, sharing the workload for various services. In this embodiment, the GPU unit and the CPU unit can be used together to process the initial image data to obtain intermediate image data. In practical applications, the CPU can also control and allocate PCIe resources.
[0040] For example, the GPU and CPU units are primarily used for image preprocessing, image enhancement, image segmentation, feature extraction, image registration, image synthesis, video processing, deep learning and artificial intelligence, real-time processing, and 3D reconstruction. It should be noted that the specific image processing tasks performed by the CPU and GPU units can be configured according to the actual application scenario; no specific restrictions are imposed here.
[0041] In some embodiments, the video processing unit further includes a field-programmable gate array (FPGA) unit, and the video processing unit is further configured to process the initial image data through the FPGA unit and the GPU unit to obtain intermediate image data.
[0042] In some embodiments, the video processing unit further includes a field-programmable gate array (FPGA) unit, and the video processing unit is also used to process the initial image data through the GPU unit, CPU unit, and FPGA unit to obtain intermediate image data.
[0043] Specifically, in Figure 2 Based on, refer to Figure 3 As shown, in addition to using the GPU unit 1021 and CPU unit 1022 to process the initial image data, the video processing unit 102 also includes an FPGA unit 1023. The video processing unit can also process the initial image data using the FPGA unit 1023 and GPU unit 1021 to obtain intermediate image data. Alternatively, the video processing unit 102 can also process the initial image data using the GPU unit 1021, CPU unit 1022, and FPGA unit 1023 to obtain intermediate image data. In this embodiment, the FPGA unit 1023 can also participate in image data processing, sharing the workload among various services. The GPU unit 1021, CPU unit 1022, and FPGA unit 1023 perform different image processing tasks according to specific application scenarios.
[0044] For example, CPU units, GPU units, and FPGA units can all process images. For instance, a GPU unit can perform image preprocessing on the initial image, such as noise reduction and edge detection. A CPU unit can perform image scaling, detail enhancement, color correction, and noise filtering on the initial image. An FPGA unit can perform gamma correction and display pixel correction on the initial image data.
[0045] In some embodiments, the video processing unit further includes a PCIe switch, and the GPU unit is connected to the CPU unit and the FPGA unit via the PCIe switch.
[0046] Specifically, in Figure 3 Based on, refer to Figure 4 As shown, the video processing unit 102 also includes a PCIe switch 1024. The GPU unit 1021 is connected to the CPU unit 1022 and the FPGA unit 1023 via the PCIe switch 1024. The video processing unit 102 internally has a PCIe switch that connects the GPU, FPGA, and CPU via a PCIe Gen4 x16 interface (PCIe Gen4 x16 indicates that the interface supports fourth-generation PCIe technology and has 16 data lanes) for data and command bridging, facilitating system performance expansion. A PCIe switch is a network switching device used to expand and connect multiple PCIe devices. It allows multiple PCIe devices to communicate with each other, improving the flexibility and efficiency of data transmission.
[0047] In some embodiments, the PCIe switch includes a PCIe interface, and the GPU unit is connected to the CPU unit and the FPGA unit through the PCIe interface.
[0048] The PCIe (Peripheral Component Interconnect Express) interface is a high-speed serial computer expansion bus standard used to connect the motherboard to various expansion devices.
[0049] Specifically, within this device, the PCIe switch can be replaced by other solutions, such as interconnecting the CPU unit, GPU unit, and FPGA unit via the PCIe interface for data transmission.
[0050] In some embodiments, the FPGA unit includes a high-speed serial transmission interface, and the GPU unit is connected to the CPU unit and the FPGA unit respectively through the high-speed serial transmission interface of the FPGA unit.
[0051] Among them, high-speed serial transmission interfaces include, but are not limited to, PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard), optical ports (i.e., fiber optic interfaces), and network ports (including Ethernet interfaces, etc.).
[0052] Specifically, within this device, the CPU unit, GPU unit, and FPGA unit are interconnected via a high-speed serial transmission interface of the FPGA unit for data transmission. Alternatively, the CPU unit, GPU unit, and FPGA unit can communicate in other reasonable ways, without specific limitations here.
[0053] In some embodiments, the first image processing device further includes a network switch for transmitting video streams to at least one second image processing device.
[0054] In this embodiment, the first image processing device 100 further includes a network switch 105. The network switch receives data packets from connected devices and forwards the received data packets to other devices connected to the same switch. In this embodiment, the network switch can send data from the first image processing device 100 to a second image processing device 200. The second image processing device 200 can have the same internal structure as the first image processing device 100 provided in this embodiment. The second image processing device 200 includes an input unit 201, a video processing unit 202, an output unit 203, and a network switch 205. It should be noted that the network switch can also be located independently outside the first image processing device for data transmission between multiple image processing devices.
[0055] In some embodiments, the network switch is also used to transmit the initial image data to at least one second image processing device.
[0056] In some embodiments, the network switch is also used to transmit intermediate image data to at least one second image processing device.
[0057] In some embodiments, the network switch is also used to transmit display data to at least one second image processing device.
[0058] Specifically, refer to Figure 5 As shown, the first image processing device 100 can send video streams, initial image data, intermediate image data, and display data to the second image processing device 200 via network exchange 105.
[0059] This design can effectively distribute the instantaneous processing pressure on nodes, improving the overall performance and reliability of the system. GPU units, FPGA units, and CPU units each have their unique advantages and applicable scenarios when processing image data. By rationally utilizing the characteristics of these architectures, the performance, flexibility, and efficiency of image processing can be significantly improved.
[0060] For example, refer to Figure 5 As shown, assuming the resolution of the image to be processed is 8K, but the first image processing device 100 can only process images with a resolution of 4K, the first image processing device 100 and the second image processing device 200 are cascaded through a network switch so that the two image processing devices process the image in parallel and send the processed display data to the display screen to improve the overall performance of the system.
[0061] By cascading multiple controllers to process the same video stream / initial image data / intermediate image data / target display image data, they can be stacked indefinitely without requiring hardware redesign. That is, the network switch can transmit not only video streams between controllers, but also initial image data, intermediate image data, or target display image data. This design can effectively distribute the instantaneous processing pressure of nodes, thereby shortening development time, meeting customer needs in a short period of time, and quickly customizing solutions according to actual application scenarios.
[0062] In some embodiments, the LED display screen includes at least one housing, each housing including at least one receiver card, the receiver card being used to drive the LED display screen to illuminate the LED display screen.
[0063] The number of enclosures can be set according to the actual application, and there is no specific limit here.
[0064] In one embodiment, in Figure 3 Expanding upon the existing foundation, referencing Figure 6 As shown, Figure 6 This is another schematic diagram of an image processing device. Taking the first image processing device 100, which includes an input unit 101, a video processing unit 102, and an output unit 103, as an example, the LED display screen 104 includes 10 cabinets, each containing at least one receiving card. The receiving card is used to drive the LED display screen to illuminate it.
[0065] In some embodiments, the image processing device includes at least one GPU unit.
[0066] The number of GPU units can be set according to the actual application scenario, and there is no specific limit here.
[0067] Specifically, system performance is improved by increasing the number of GPU units to process the same image.
[0068] The image processing device provided by this utility model includes an input unit, a video processing unit, and an output unit. The input unit is connected to the video processing unit, the video processing unit is connected to the output unit, and the output unit is connected to an LED display screen. The input unit is used to acquire a video stream, decode the video stream to obtain initial image data, and send the initial image data to the video processing unit. The video processing unit is used to process the initial image data to obtain intermediate image data and send the intermediate image data to the output unit. The video processing unit includes a graphics processing unit (GPU). Specifically, the video processing unit processes the initial image data through the GPU to obtain intermediate image data. The output unit processes the intermediate image data to obtain display data and sends the display data to the LED display screen for display. During image processing, because the GPU unit focuses on parallel processing of image data and can perform complex image processing algorithms, processing the initial image through the GPU unit reduces the workload compared to relying solely on the FPGA for image processing. Furthermore, because the GPU can perform image processing through machine learning algorithms, and the processing algorithm accuracy of the GPU is higher than that of traditional FPGA fixed-point image processing, it can improve the display quality.
[0069] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0070] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An image processing device, characterized in that, The first image processing device includes: an input unit, a video processing unit, and an output unit, wherein the input unit is connected to the video processing unit, the video processing unit is connected to the output unit, and the output unit is connected to an LED display screen; The input unit is used to acquire a video stream, decode the video stream to obtain initial image data, and send the initial image data to the video processing unit; The video processing unit is used to process the initial image data to obtain intermediate image data, and send the intermediate image data to the output unit; The video processing unit includes a graphics processing unit (GPU); specifically, the video processing unit is used to process the initial image data through the GPU to obtain intermediate image data. The output unit is used to process the intermediate image data to obtain display data, and then send the display data to the LED display screen for display.
2. The device according to claim 1, characterized in that, The video processing unit further includes a central processing unit (CPU) unit, and the video processing unit is also used to process the initial image data through the CPU unit and the GPU unit to obtain intermediate image data.
3. The device according to claim 1, characterized in that, The video processing unit further includes a field-programmable gate array (FPGA) unit, and the video processing unit is also used to process the initial image data through the FPGA unit and the GPU unit to obtain intermediate image data.
4. The device according to claim 2, characterized in that, The video processing unit further includes a field-programmable gate array (FPGA) unit, and the video processing unit is also used to process the initial image data through the GPU unit, the CPU unit, and the FPGA unit to obtain intermediate image data.
5. The device according to claim 4, characterized in that, The video processing unit also includes a PCIe switch, and the GPU unit is connected to the CPU unit and the FPGA unit through the PCIe switch.
6. The device according to claim 5, characterized in that, The PCIe switch includes PCIe interfaces, and the GPU unit is connected to the CPU unit through the PCIe interfaces.
7. The device according to claim 1, characterized in that, The device also includes a network switch for transmitting the video stream to at least one second image processing device.
8. The device according to claim 7, characterized in that, The network switch is also used to transmit the initial image data to at least one second image processing device.
9. The device according to claim 7, characterized in that, The network switch is also used to transmit the intermediate image data to at least one second image processing device.
10. The device according to claim 7, characterized in that, The network switch is also used to transmit the display data to at least one second image processing device.
11. The device according to claim 1, characterized in that, The LED display screen includes at least one cabinet, and each cabinet includes at least one receiver card, which is used to drive the LED display screen to light up the LED display screen.
12. The device according to claim 1, characterized in that, The device includes at least one GPU unit.