Image processing method and apparatus, and electronic device and storage medium
By combining image processing methods with GPU and FPGA units, the problems of unclear image quality and high development difficulty in existing LED image processing are solved, achieving efficient and high-definition image display effects.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- XIAN NOVASTAR TECH
- Filing Date
- 2025-12-30
- Publication Date
- 2026-07-09
Smart Images
Figure CN2025147194_09072026_PF_FP_ABST
Abstract
Description
Image processing methods, apparatus, electronic devices and storage media
[0001] This disclosure claims priority to Chinese Patent Application No. 2024119980518, filed with the Chinese Patent Office on December 31, 2024, entitled "Image Processing Method, Apparatus, Electronic Device and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field
[0002] This disclosure relates to the field of LED technology, and in particular to an image processing method, apparatus, electronic device and storage medium. Background Technology
[0003] Currently, image processing solutions in the LED (Light Emitting Diode) industry primarily rely on FPGA (Field-Programmable Gate Array) designs. For example, FPGA-based heterogeneous computing can quickly build high-performance image processing solutions, and by hardware-enabling image processing algorithms, it can meet the requirements for real-time performance and high performance. However, with the development of LED technology and the increasing market demands for image processing, using only FPGAs for image processing results in insufficient image quality. Furthermore, FPGA-based image processing in the LED field faces challenges such as long development cycles, high complexity, and limited resources. Summary of the Invention
[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides an image processing method, apparatus, electronic device, and storage medium, which solves the problems that the image quality is not clear enough when using FPGA to perform heterogeneous computing processing on images, and that FPGA has problems such as long development cycle, high difficulty, and limited resources when performing image processing in the LED field.
[0005] To achieve the above objectives, the present disclosure provides the following technical solutions:
[0006] In a first aspect, embodiments of this disclosure provide an image processing method applied to a first control device, the first control device comprising: an input unit, a video processing unit, and an output unit, wherein the input unit is connected to the video processing unit, and the video processing unit is connected to the output unit; the method comprises:
[0007] The video stream is acquired through the input unit, the video stream is decoded to obtain initial image data, and the initial image data is sent to the video processing unit.
[0008] The initial image data is processed by the video processing unit to obtain intermediate image data, and the intermediate image data is sent to the output unit; wherein, the video processing unit includes a graphics processing unit (GPU).
[0009] The step of processing the initial image data through the video processing unit to obtain intermediate image data includes:
[0010] The GPU unit processes the initial image data to obtain intermediate image data;
[0011] The output unit processes the intermediate image data to obtain display data, and sends the display data to the receiving device so that the receiving device drives the display screen to display the image.
[0012] As an optional implementation of this disclosure, the video processing unit further includes a central processing unit (CPU) unit. The step of processing the initial image data through the video processing unit to obtain intermediate image data includes:
[0013] The CPU unit processes the initial image data to obtain intermediate image data.
[0014] As an optional implementation of this disclosure, the video processing unit further includes a field-programmable gate array (FPGA) unit. The step of processing the initial image data through the video processing unit to obtain intermediate image data includes:
[0015] The initial image data is processed by the FPGA unit to obtain intermediate image data.
[0016] As an optional implementation of this disclosure, the step of processing the intermediate image data through the output unit to obtain display data, and sending the display data to the receiving device so that the receiving device drives the display screen to display, includes:
[0017] The output unit performs format conversion processing on the intermediate image data to obtain display data.
[0018] The display data is sent to the receiving device so that the receiving device drives the display screen to display the display data.
[0019] As an optional implementation of this disclosure, the input unit includes a field-programmable gate array (FPGA) unit; the acquisition of the video stream through the input unit includes:
[0020] The video stream is obtained through the video interface of the FPGA unit;
[0021] or;
[0022] The high-speed serial transmission interface of the FPGA unit is connected to a network switch, and the video stream is obtained through the network switch.
[0023] As an optional implementation of this disclosure, the method further includes:
[0024] The video stream is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0025] As an optional implementation of this disclosure, the method further includes:
[0026] The initial image data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0027] As an optional implementation of this disclosure, the method further includes:
[0028] The intermediate image data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0029] As an optional implementation of this disclosure, the method further includes:
[0030] The display data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0031] In a second aspect, embodiments of this disclosure provide an image processing apparatus, comprising:
[0032] The acquisition module is used to acquire a video stream through the input unit, decode the video stream to obtain initial image data, and send the initial image data to the video processing unit;
[0033] A processing module is configured to process the initial image data through the video processing unit to obtain intermediate image data, and send the intermediate image data to the output unit; wherein, the video processing unit includes a GPU unit;
[0034] The processing module is specifically used for:
[0035] The GPU unit processes the initial image data to obtain intermediate image data;
[0036] The display module is used to process the intermediate image data through the output unit to obtain display data, and send the display data to the receiving device so that the receiving device drives the display screen to display.
[0037] As an optional implementation of this disclosure, the video processing unit further includes a central processing unit (CPU), and the processing module is specifically used for:
[0038] The CPU unit processes the initial image data to obtain intermediate image data.
[0039] As an optional implementation of this disclosure, the video processing unit further includes a field-programmable gate array (FPGA) unit, and the processing module is specifically used for:
[0040] The initial image data is processed by the FPGA unit to obtain intermediate image data.
[0041] As an optional implementation of this disclosure, the display module is specifically used for:
[0042] The output unit performs format conversion processing on the intermediate image data to obtain display data.
[0043] The display data is sent to the receiving device so that the receiving device drives the display screen to display the display data.
[0044] As an optional implementation of this disclosure, the input unit includes a field-programmable gate array (FPGA) unit; the acquisition module is specifically used for:
[0045] The video stream is obtained through the video interface of the FPGA unit;
[0046] or;
[0047] The high-speed serial transmission interface of the FPGA unit is connected to a network switch, and the video stream is obtained through the network switch.
[0048] As an optional implementation of this disclosure, the apparatus further includes: a cascading module for transmitting the video stream to at least one second control device via the video interface of the FPGA unit or the network switch.
[0049] As an optional implementation of this disclosure, the cascade module is further configured to:
[0050] The initial image data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0051] As an optional implementation of this disclosure, the cascade module is further configured to:
[0052] The intermediate image data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0053] As an optional implementation of this disclosure, the cascading module is further configured to: send the display data to at least one second control device via the video interface of the FPGA unit or the network switch.
[0054] Thirdly, embodiments of this disclosure provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the image processing method described in the first aspect or any embodiment of the first aspect.
[0055] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method described in the first aspect or any embodiment of the first aspect.
[0056] The image processing method disclosed herein is applied to a first control device, which includes an input unit, a video processing unit, and an output unit. The input unit and the video processing unit are connected, and the video processing unit and the output unit are also connected. The input unit acquires a video stream, decodes the video stream to obtain initial image data, and sends the initial image data to the video processing unit. The video processing unit processes the initial image data to obtain intermediate image data, which is then sent to the output unit. The video processing unit includes a GPU unit. The GPU unit processes the initial image data to obtain intermediate image data, and the output unit processes the intermediate image data to obtain display data, which is then sent to a receiving device so that the receiving device drives the display screen for display. During image processing, because the GPU unit focuses on parallel processing of image data and can perform complex image processing algorithms, processing the initial image using the GPU unit reduces the workload compared to relying solely on the FPGA for image processing. Furthermore, since the GPU can perform image processing using machine learning algorithms, and the processing accuracy of the GPU is higher than that of traditional FPGA fixed-point image processing, it can improve display quality.
[0057] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0058] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0059] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0060] Figure 1 is a flowchart illustrating an image processing method in one embodiment;
[0061] Figure 2 is a schematic diagram of an application scenario of the image processing method in one embodiment;
[0062] Figure 3 is a schematic diagram of the structure of an image processing device in one embodiment;
[0063] Figure 4 is a schematic diagram of the structure of the electronic device described in an embodiment of this disclosure. Detailed Implementation
[0064] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.
[0065] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.
[0066] The terms "first" and "second" and other relational terms used in this disclosure and claims are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations.
[0067] In this disclosure, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. Furthermore, in the description of the embodiments in this disclosure, unless otherwise stated, "a plurality of" means two or more.
[0068] Currently, many image processing solutions rely primarily on Field Programmable Gate Array (FPGA) designs, with multiple FPGAs cascaded to process image content. This is suitable for various application scenarios. However, with technological advancements and increasing market demands for image processing, particularly for high-resolution, high-quality images and real-time processing, existing solutions are beginning to face challenges. These challenges include long FPGA development cycles, limited internal resources, high development difficulty, and insufficient flexibility.
[0069] To address the aforementioned problems, this disclosure provides an image processing method applied to a first control device. The first control device includes an input unit, a video processing unit, and an output unit. The input unit and the video processing unit are connected, and the video processing unit and the output unit are also connected. The input unit acquires a video stream, decodes the video stream to obtain initial image data, and sends the initial image data to the video processing unit. The video processing unit processes the initial image data to obtain intermediate image data, which is then sent to the output unit. The video processing unit includes a GPU unit. The GPU unit processes the initial image data to obtain intermediate image data, and the output unit processes the intermediate image data to obtain display data, which is then sent to a receiving device so that the receiving device drives the display screen for display. During image processing, because the GPU unit focuses on parallel processing of image data and can perform complex image processing algorithms, processing the initial image using the GPU unit reduces the workload compared to relying solely on the FPGA for image processing. Furthermore, since the GPU can perform image processing using machine learning algorithms, and the accuracy of GPU processing algorithms is higher than that of traditional FPGA fixed-point image processing, it can improve display quality.
[0070] In one embodiment, as shown in Figure 1, an image processing method is provided. This method can be applied to image processing scenarios in the LED field. In this scenario, a video stream is input to a first control device, which processes the video stream and displays the processed data on an LED display screen. The first control device includes an input unit, a video processing unit, and an output unit. The input unit is connected to the video processing unit, and the video processing unit is connected to the output unit. The image processing method includes the following steps:
[0071] S11. Obtain the video stream through the input unit, decode the video stream to obtain initial image data, and send the initial image data to the video processing unit.
[0072] The input unit can be a field-programmable gate array (FPGA) unit, or other units that can acquire and decode video streams; no specific restrictions are imposed here.
[0073] Specifically, the video stream can be acquired through an FPGA unit, decoded, and processed to obtain a video signal. A video signal refers to image information transmitted electronically; it is a continuous analog or digital signal used to represent image data. That is, after acquiring the video stream through the FPGA unit, it is decoded to obtain initial image data. The initial image data includes RGB data, pixel data, synchronization signals, timing information, field signals, etc. Decoding methods include, but are not limited to, H.264 / AVC video decoding (an advanced video coding standard widely used in digital video transmission and storage) and H.265 video compression and decoding (a video compression standard).
[0074] In some embodiments, the input unit includes a field-programmable gate array (FPGA) unit, and step S11 (acquiring a video stream through the input unit) includes:
[0075] The video stream is obtained through the video interface of the FPGA unit;
[0076] or;
[0077] The high-speed serial transmission interface of the FPGA unit is connected to a network switch, and the video stream is obtained through the network switch.
[0078] Among them, video interfaces include, but are not limited to, HDMI (High-Definition Multimedia Interface) and DP (DisplayPort).
[0079] High-speed serial transmission interfaces include, but are not limited to, PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard), optical ports (i.e., fiber optic interfaces), and network ports (including Ethernet interfaces, etc.).
[0080] A network switch is used to receive data packets from connected devices and forward the received data packets to other devices connected to the same switch. In this embodiment of the disclosure, the network switch can send data from a first control device to a second control device.
[0081] Specifically, the video stream is acquired through the video interface of the FPGA unit on the first control device, or the Ethernet interface of the FPGA unit is connected to a network switch to acquire the video stream through the network switch. The FPGA also has a high-speed serial transmission interface, which, through the network switch, is used to receive or share the image data of the current device with other devices.
[0082] S12. The initial image data is processed by the video processing unit to obtain intermediate image data, and the intermediate image data is sent to the output unit.
[0083] The video processing unit includes a graphics processing unit (GPU).
[0084] Optionally, step S12 (processing the initial image data by the video processing unit to obtain intermediate image data) can be implemented in the following way:
[0085] The GPU unit processes the initial image data to obtain intermediate image data.
[0086] Specifically, GPUs, with their numerous parallel processing units, are well-suited for handling large-scale parallel computing tasks such as convolutional neural networks (CNNs) and image filtering. Furthermore, GPUs have high-bandwidth video memory, making them suitable for processing large amounts of data. Meanwhile, NVIDIA and AMD (graphics processing and related technology companies) provide abundant development tools and libraries, such as CUDA (Compute Unified Device Architecture, a general-purpose parallel computing platform and programming model), OpenCL (Open Computing Language), and cuDNN (CUDA Deep Neural Network, a GPU-accelerated library for deep neural networks), offering a wealth of open-source libraries and community support, facilitating rapid development. Therefore, by using GPU units to process initial image data, intermediate image data is obtained and sent to the output unit. Utilizing GPUs for image processing can significantly shorten the development cycle, reduce development difficulty, and improve processing speed and efficiency.
[0087] In some embodiments, the video processing unit further includes a central processing unit (CPU), and the above-mentioned step S12 (processing the initial image data by the video processing unit to obtain intermediate image data) can also be implemented in the following manner:
[0088] The CPU unit processes the initial image data to obtain intermediate image data.
[0089] Specifically, because the CPU is a general-purpose processor, it is suitable for various computing tasks, including control logic and complex algorithms in image processing. Furthermore, the CPU can be programmed using high-level languages (such as C / C++, Python), resulting in a short development cycle and ease of maintenance; it has extensive development tools and library support, such as OpenCV and NumPy, facilitating quick learning for developers; and it can handle multiple tasks simultaneously, making it suitable for scenarios requiring multi-threading and multi-tasking. Therefore, the CPU unit can process the initial image data to obtain intermediate image data. That is, the CPU also participates in image data processing, sharing the workload among various business processes. In this embodiment, the GPU unit and the CPU unit can be used together to process the initial image data to obtain intermediate image data. In practical applications, the CPU can also control and allocate PCIe resources.
[0090] For example, the GPU and CPU units are primarily used for image preprocessing, image enhancement, image segmentation, feature extraction, image registration, image synthesis, video processing, deep learning and artificial intelligence, real-time processing, and 3D reconstruction. It should be noted that the specific image processing tasks performed by the CPU and GPU units can be configured according to the actual application scenario; no specific restrictions are imposed here.
[0091] In some embodiments, the video processing unit further includes a field-programmable gate array (FPGA) unit, and the above step S12 (processing the initial image data by the video processing unit to obtain intermediate image data) can also be implemented in the following manner:
[0092] The initial image data is processed by the FPGA unit to obtain intermediate image data.
[0093] Specifically, in addition to using GPU and CPU units to process the initial image data, it can also be processed using GPU and FPGA units, or a combination of GPU, CPU, and FPGA units to obtain intermediate image data. That is, FPGA units can also participate in image data processing, relieving pressure on various business processes. GPU, CPU, and FPGA units perform different image processing tasks depending on the specific application scenario.
[0094] For example, in the first control device, the CPU unit, GPU unit, and FPGA unit can all process images. For instance, the GPU unit can perform image preprocessing on the initial image, such as noise reduction and edge detection. The CPU unit can perform image scaling, detail enhancement, color correction, and noise filtering on the initial image. The FPGA unit can perform gamma correction and display pixel correction on the initial image data.
[0095] S13. The intermediate image data is processed by the output unit to obtain display data, and the display data is sent to the receiving device so that the receiving device drives the display screen to display.
[0096] Optionally, step S13 (processing the intermediate image data through the output unit to obtain display data, and sending the display data to the receiving device so that the receiving device drives the display screen to display) can be implemented in the following way:
[0097] The output unit performs format conversion processing on the intermediate image data to obtain display data.
[0098] The display data is sent to the receiving device so that the receiving device drives the display screen to display the display data.
[0099] Specifically, the output unit may include, but is not limited to, FPGA units, GPU units, and CPU units. The output unit performs format conversion processing on the intermediate image data to obtain display data. The output unit can perform conversions between different video formats, such as converting RGB video to YUV format, or converting videos of different resolutions and frame rates. According to the requirements of the backend equipment, it performs format conversion on the video signal to ensure that the video can be displayed and played correctly. The display data is then sent to the receiving device so that the receiving device can drive the display screen to display the data.
[0100] According to embodiments of this disclosure, the image processing method provided is applied to a first control device. The first control device includes an input unit, a video processing unit, and an output unit. The input unit and the video processing unit are connected, and the video processing unit and the output unit are also connected. The input unit acquires a video stream, decodes the video stream to obtain initial image data, and sends the initial image data to the video processing unit. The video processing unit processes the initial image data to obtain intermediate image data, and sends the intermediate image data to the output unit. The video processing unit includes a GPU unit. The GPU unit processes the initial image data to obtain intermediate image data, and the output unit processes the intermediate image data to obtain display data, which is then sent to a receiving device so that the receiving device drives the display screen to display the image. During image processing, because the GPU unit focuses on parallel processing of image data and can perform complex image processing algorithms, processing the initial image using the GPU unit reduces the workload compared to relying solely on the FPGA for image processing. Furthermore, because the GPU can perform image processing using machine learning algorithms, and the processing algorithm accuracy of the GPU is higher than that of traditional FPGA fixed-point image processing, it can improve display quality.
[0101] In some embodiments, the video stream is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0102] Specifically, the video stream is sent to at least one second control device via the video interface of the FPGA unit or a network switch.
[0103] For example, referring to Figure 2, which is a schematic diagram of an application scenario of an image processing method in one embodiment, the first control device 101 and the second control device 103 communicate through a network switch 104; or, the first control device 101 and the second control device 103 communicate through the video interface of FPGA unit 1013 and the video interface of FPGA unit 1033. The first control device 101 includes a CPU unit 1011, a GPU unit 1012, and an FPGA unit 1013, and the second control device 103 includes a CPU unit 1031, a GPU unit 1032, and an FPGA unit 1033. The first control device 101 and the second control device 103 are communicatively connected through FPGA unit 1013 and FPGA unit 1033. The network switch is used to receive data packets from connected devices and forward them to other devices connected to the same switch.
[0104] In this embodiment of the disclosure, referring to FIG2, the first control device 101 and the second control device 103 can communicate data in two ways. The first way is to send the video stream acquired by the first control device to the second control device through the network switch 104; the second way is to send the video stream acquired by the first control device to the second control device through the video interface of the FPGA unit. For example, the video stream is sent to the video interface of the FPGA unit 1033 of the second control device 103 through the video interface of the FPGA unit 1013 of the first control device 101.
[0105] In some embodiments, the initial image data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0106] In this embodiment of the disclosure, referring to FIG2, the first control device 101 and the second control device 103 can communicate data in two ways. The first way is to send the initial image data of the first control device 101 to the second control device 103 through the network switch 104; the second way is to send the initial image data to the video interface of the FPGA unit 1033 of the second control device 103 through the video interface of the FPGA unit 1013.
[0107] In some embodiments, the intermediate image data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0108] In this embodiment of the disclosure, referring to FIG2, the first control device 101 and the second control device 103 can communicate data in two ways. The first way is to send the intermediate image data of the first control device 101 to the second control device 103 through the network switch 104; the second way is to send the intermediate image data to the video interface of the FPGA unit 1033 of the second control device 103 through the video interface of the FPGA unit 1013.
[0109] In some embodiments, the display data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0110] In this embodiment of the disclosure, referring to FIG2, the first control device 101 and the second control device 103 can communicate data in two ways. The first way is to send the display data of the first control device 101 to the second control device 103 through the network switch 104; the second way is to send the display data to the video interface of the FPGA unit 1033 of the second control device 103 through the video interface of the FPGA unit 1013.
[0111] This design effectively distributes the instantaneous processing load of nodes, improving the overall performance and reliability of the system. Through this multi-level cascading approach, the same video stream can be processed indefinitely without requiring hardware redesign.
[0112] In the embodiments disclosed herein, the GPU unit, FPGA unit, and CPU unit each have their unique advantages and applicable scenarios when processing image data. By making reasonable use of the characteristics of these units, the performance, flexibility, and efficiency of image processing can be significantly improved.
[0113] According to embodiments of this disclosure, the image processing method provided is applied to a first control device. The first control device includes an input unit, a video processing unit, and an output unit. The input unit and the video processing unit are connected, and the video processing unit and the output unit are also connected. The input unit acquires a video stream, decodes the video stream to obtain initial image data, and sends the initial image data to the video processing unit. The video processing unit processes the initial image data to obtain intermediate image data, and sends the intermediate image data to the output unit. The video processing unit includes a GPU unit. The GPU unit processes the initial image data to obtain intermediate image data, and the output unit processes the intermediate image data to obtain display data, which is then sent to a receiving device so that the receiving device drives the display screen to display the image. During image processing, because the GPU unit focuses on parallel processing of image data and can perform complex image processing algorithms, processing the initial image using the GPU unit reduces the workload compared to relying solely on the FPGA for image processing. Furthermore, because the GPU can perform image processing using machine learning algorithms, and the processing algorithm accuracy of the GPU is higher than that of traditional FPGA fixed-point image processing, it can improve display quality.
[0114] In one embodiment, as shown in FIG3, an image processing apparatus 300 is provided, comprising:
[0115] The acquisition module 310 is used to acquire a video stream through the input unit, decode the video stream to obtain initial image data, and send the initial image data to the video processing unit.
[0116] The processing module 320 is used to process the initial image data through the video processing unit to obtain intermediate image data, and send the intermediate image data to the output unit; wherein, the video processing unit includes a GPU unit;
[0117] The processing module 320 is specifically used for:
[0118] The GPU unit processes the initial image data to obtain intermediate image data;
[0119] The display module 330 is used to process the intermediate image data through the output unit to obtain display data, and send the display data to the receiving device so that the receiving device drives the display screen to display.
[0120] As an optional implementation of this disclosure, the video processing unit further includes a central processing unit (CPU), and the processing module is specifically used for:
[0121] The CPU unit processes the initial image data to obtain intermediate image data.
[0122] As an optional implementation of this disclosure, the video processing unit further includes a field-programmable gate array (FPGA) unit, and the processing module is specifically used for:
[0123] The initial image data is processed by the FPGA unit to obtain intermediate image data.
[0124] As an optional implementation of this disclosure, the display module is specifically used for:
[0125] The output unit performs format conversion processing on the intermediate image data to obtain display data.
[0126] The display data is sent to the receiving device so that the receiving device drives the display screen to display the display data.
[0127] As an optional implementation of this disclosure, the input unit includes a field-programmable gate array (FPGA) unit; the acquisition module is specifically used for:
[0128] The video stream is obtained through the video interface of the FPGA unit;
[0129] or;
[0130] The high-speed serial transmission interface of the FPGA unit is connected to a network switch, and the video stream is obtained through the network switch.
[0131] As an optional implementation of this disclosure, the apparatus further includes: a cascading module for transmitting the video stream to at least one second control device via the video interface of the FPGA unit or the network switch.
[0132] As an optional implementation of this disclosure, the cascade module is further configured to:
[0133] The initial image data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0134] As an optional implementation of this disclosure, the cascade module is further configured to:
[0135] The intermediate image data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
[0136] As an optional implementation of this disclosure, the cascading module is further configured to: send the display data to at least one second control device via the video interface of the FPGA unit or the network switch.
[0137] According to embodiments of this disclosure, the image processing apparatus provided is applied to a first control device. The first control device includes an input unit, a video processing unit, and an output unit. The input unit and the video processing unit are connected, and the video processing unit and the output unit are also connected. The input unit acquires a video stream, decodes the video stream to obtain initial image data, and sends the initial image data to the video processing unit. The video processing unit processes the initial image data to obtain intermediate image data, and sends the intermediate image data to the output unit. The video processing unit includes a GPU unit. The GPU unit processes the initial image data to obtain intermediate image data, and the output unit processes the intermediate image data to obtain display data, which is then sent to a receiving device so that the receiving device drives the display screen to display the image. During image processing, because the GPU unit focuses on parallel processing of image data and can perform complex image processing algorithms, processing the initial image using the GPU unit reduces the workload compared to relying solely on the FPGA for image processing. Furthermore, because the GPU can perform image processing using machine learning algorithms, and the processing algorithm accuracy of the GPU is higher than that of traditional FPGA fixed-point image processing, it can improve display quality.
[0138] For specific limitations regarding the image processing device, please refer to the limitations on the image processing method above, which will not be repeated here. Each module in the aforementioned image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor of the electronic device, or stored in software within the processor of the electronic device, so that the processor can call and execute the operations corresponding to each module.
[0139] This disclosure also provides an electronic device, and Figure 4 is a schematic diagram of the structure of the electronic device provided in this disclosure embodiment. As shown in Figure 4, the electronic device provided in this embodiment includes: a memory 41 and a processor 42. The memory 41 is used to store computer programs; the processor 42 is used to execute the steps of any embodiment of the image processing method provided in the above method embodiments when the computer program is invoked. The electronic device includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. The processor of the electronic device provides computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. When the computer program is executed by the processor, it implements an image processing method. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer covering the display screen, or a button, trackball, or touchpad provided on the casing of a computer device, or an external keyboard, touchpad, or mouse, etc.
[0140] Those skilled in the art will understand that the structure shown in Figure 4 is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the computer device to which the present disclosure is applied. Specific electronic devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0141] In one embodiment, the image processing apparatus provided in this disclosure can be implemented as a computer, and the computer program can run on the electronic device shown in FIG4. The memory of the electronic device can store various program modules constituting the client-type image processing apparatus of the electronic device. The computer program composed of the various program modules causes the processor to execute the steps in the image processing methods of the electronic device according to the various embodiments of this disclosure described herein.
[0142] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image processing method provided in the above-described method embodiments.
[0143] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code.
[0144] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.
[0145] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0146] Computer-readable media include both permanent and non-permanent, removable and non-removable storage media. Storage media can store information using any method or technology; the information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media do not include transient computer-readable media, such as modulated data signals and carrier waves.
[0147] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0148] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An image processing method, characterized in that, The method is applied to a first control device, the first control device comprising: an input unit, a video processing unit, and an output unit, wherein the input unit is connected to the video processing unit, and the video processing unit is connected to the output unit; the method includes: The video stream is acquired through the input unit, the video stream is decoded to obtain initial image data, and the initial image data is sent to the video processing unit. The initial image data is processed by the video processing unit to obtain intermediate image data, and the intermediate image data is sent to the output unit; wherein, the video processing unit includes a graphics processing unit (GPU). The step of processing the initial image data through the video processing unit to obtain intermediate image data includes: The GPU unit processes the initial image data to obtain intermediate image data; The output unit processes the intermediate image data to obtain display data, and sends the display data to the receiving device so that the receiving device drives the display screen to display the image.
2. The method according to claim 1, characterized in that, The video processing unit further includes a central processing unit (CPU) unit. The process of processing the initial image data through the video processing unit to obtain intermediate image data includes: The CPU unit processes the initial image data to obtain intermediate image data.
3. The method according to claim 1 or 2, characterized in that, The video processing unit further includes a field-programmable gate array (FPGA) unit. The process of processing the initial image data through the video processing unit to obtain intermediate image data includes: The initial image data is processed by the FPGA unit to obtain intermediate image data.
4. The method according to claim 1, characterized in that, The step of processing the intermediate image data through the output unit to obtain display data, and sending the display data to the receiving device so that the receiving device drives the display screen to display, includes: The output unit performs format conversion processing on the intermediate image data to obtain display data. The display data is sent to the receiving device so that the receiving device drives the display screen to display the display data.
5. The method according to claim 1, characterized in that, The input unit includes a field-programmable gate array (FPGA) unit; the acquisition of the video stream through the input unit includes: The video stream is obtained through the video interface of the FPGA unit; or; The high-speed serial transmission interface of the FPGA unit is connected to a network switch, and the video stream is obtained through the network switch.
6. The method according to claim 5, characterized in that, The method further includes: The video stream is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
7. The method according to claim 5, characterized in that, The method further includes: The initial image data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
8. The method according to claim 5, characterized in that, The method further includes: The intermediate image data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
9. The method according to claim 5, characterized in that, The method further includes: The display data is sent to at least one second control device via the video interface of the FPGA unit or the network switch.
10. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire a video stream through the input unit, decode the video stream to obtain initial image data, and send the initial image data to the video processing unit; A processing module is configured to process the initial image data through the video processing unit to obtain intermediate image data, and send the intermediate image data to the output unit; wherein, the video processing unit includes a GPU unit; The processing module is specifically used for: The GPU unit processes the initial image data to obtain intermediate image data; The display module is used to process the intermediate image data through the output unit to obtain display data, and send the display data to the receiving device so that the receiving device drives the display screen to display.
11. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the image processing method according to any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the image processing method according to any one of claims 1 to 9.