Image processing method and system and medium

Abstract

The invention relates to an image processing method and system and a medium, and the method comprises the following steps: an ARM main control chip splits a to-be-displayed high-resolution original image into N paths of sub-image signals according to a target display resolution and a single-path maximum support resolution of a rear-end FPGA chip, and transmits the N paths of sub-image signals to a signal conversion chip, N is an integer greater than 1, and N is an integer greater than 1; the resolution of each sub-image does not exceed the single-path maximum support resolution of the FPGA chip; the signal conversion chip converts the N paths of sub-image signals into N paths of image signals in a format which can be supported by the rear-end FPGA chip and transmits the N paths of image signals to the FPGA chip; and the FPGA chip splices the received N paths of image signals into a complete image frame with the resolution consistent with that of the original image according to preset layout information, and then outputs the complete image frame. According to the invention, intelligent image splitting is carried out on the front-end ARM chip, and accurate image splicing is carried out on the rear-end FPGA chip, so that ultra-high-definition and high-resolution display of the LED screen is realized on the premise of not upgrading the hardware of the rear-end chip.

Description

Technical Field

[0001] This invention relates to the fields of image processing and LED display control technology, and in particular to an image method, system and medium for achieving high-resolution LED display through image splitting and splicing. Background Technology

[0002] In the existing LED display system architecture, the signal processing flow is generally adopted as "ARM main control chip → signal conversion chip → FPGA (Field Programmable Gate Array) / dedicated processing chip → LED receiver card". Specifically, the ARM main control chip (e.g., RK3588) outputs HDMI signal to the signal conversion chip (e.g., GSV2015), the signal conversion chip outputs (RGB) image signal to the FPGA chip (e.g., PH1A400SFG900), and the FPGA chip outputs image signal to the receiver card through gigabit network port to control the LED screen display.

[0003] With technological advancements, some high-performance ARM chips have the capability to output high-resolution image signals such as 8K (7680×4320). However, the final display resolution of the system is often limited by the processing power of the backend FPGA chip. Many cost-effective FPGA chips have an upper limit on the resolution of a single input signal (e.g., supporting up to 4K@60Hz), meaning that even if the ARM outputs an 8K signal, the entire system cannot achieve a display resolution exceeding 4K, forming a significant technical bottleneck.

[0004] Existing solutions typically involve directly replacing the back-end chip with one that supports higher single-channel input resolution, but this significantly increases hardware costs. Therefore, there is an urgent need in this field for an innovative software and logic algorithm solution that can overcome the resolution limitations of back-end chips on existing hardware platforms, enabling cost-effective and efficient implementation of high-resolution LED displays. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides an image processing method, system, and medium for achieving high-resolution LED display through image segmentation and stitching. It aims to overcome the hardware limitations of existing technologies by performing intelligent image segmentation on a front-end ARM chip and precise image stitching on a back-end FPGA chip, thereby achieving ultra-high-definition high-resolution LED screen display without upgrading the back-end chip hardware.

[0006] To achieve the above objectives, this invention proposes an image processing method applied to an image processing system, the image processing system comprising an ARM main control chip, a signal conversion chip, and an FPGA chip; the method includes the following steps: Step S10: The ARM main control chip splits the high-resolution original image to be displayed into N sub-image signals according to the target display resolution and the maximum single-channel supported resolution of the back-end FPGA chip, and transmits them to the signal conversion chip. Here, N is an integer greater than 1, and the resolution of each sub-image does not exceed the maximum single-channel supported resolution of the FPGA chip. Step S20: The signal conversion chip converts the N sub-image signals into N image signals in a format supported by the back-end FPGA chip and transmits them to the FPGA chip. In step S30, the FPGA chip, based on preset layout information, stitches together the received N-channel image signals into a complete image frame with the same resolution as the original image and then outputs it.

[0007] A further technical solution of the present invention is that, in step S10, the step of the ARM main control chip splitting the high-resolution original image to be displayed into N sub-image signals according to the target display resolution and the maximum single-channel supported resolution of the back-end FPGA chip includes: The ARM main control chip dynamically selects the splitting direction based on the aspect ratio of the original image. When the width is greater than the height, it splits horizontally; when the height is greater than the width, it splits vertically.

[0008] A further technical solution of the present invention is that step S20 includes: The signal conversion chip converts the HDMI signal output by the ARM main control chip into a parallel signal in RGB TTL format.

[0009] A further technical solution of the present invention is that, before step S30, the invention further includes: the ARM main control chip and the back-end FPGA chip transmitting the layout information through a communication interface, wherein the layout information includes at least the starting coordinates, width and height of each sub-image in the complete image frame.

[0010] A further technical solution of the present invention is that the communication interface is an SPI serial peripheral interface.

[0011] A further technical solution of the present invention is that the FPGA chip is internally configured with a canvas synthesizer for configuring the display area of ​​each screen separately.

[0012] A further technical solution of the present invention is that the canvas synthesizer is configured to receive multiple image inputs and configure independent display layer and display area parameters for each input to synthesize the final image on an internal virtual canvas.

[0013] A further technical solution of the present invention is that the ARM main control chip is configured with multiple independent video output ports, and each video output port is configured to read different sub-region data from the frame buffer of the original image.

[0014] To achieve the above objectives, the present invention also proposes an image processing system, the system comprising an ARM main control chip, a signal conversion chip, an FPGA chip, a memory, and a processor, wherein the processor stores an image processing program, and the image processing program is executed by the processor to perform the steps of the method described above.

[0015] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing an image processing program, which, when executed by a processor, performs the steps of the method described above.

[0016] The beneficial effects of the image processing method, system, and medium of the present invention are: 1. Breaking through hardware bottlenecks: The method of splitting and splicing LEDs to achieve high-resolution display is creatively achieved, making full use of chip capabilities and effectively avoiding the single-channel resolution limitation of the back-end FPGA chip. 2. Significantly reduced costs: Without replacing existing FPGA chips or other core hardware, the system performance can be significantly improved through algorithm upgrades, resulting in extremely high economic efficiency.

[0017] 3. Intelligent and flexible adaptation: The image splitting strategy can be automatically adjusted according to the aspect ratio of the displayed content, enhancing the system's adaptability to different application scenarios (such as ultra-wide screens and portrait screens).

[0018] 4. Stable and reliable system: Based on mature SPI communication and FPGA image synthesis technology, the solution is highly stable and easy to implement and promote. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a preferred embodiment of the image processing method of the present invention; Figure 2 This is a schematic diagram of the system framework involved in the image processing method of the present invention; Figure 3 This is a hardware architecture diagram of the image processing system of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0021] This invention proposes an image processing method, which is applied to, for example... Figure 1The image processing system shown includes an ARM main control chip, a signal conversion chip, and an FPGA chip.

[0022] like Figure 2 As shown, the preferred embodiment of the image processing method in this example includes the following steps: Step S10: The ARM main control chip splits the high-resolution original image to be displayed into N sub-image signals according to the target display resolution and the maximum single-channel supported resolution of the back-end FPGA chip, and transmits them to the signal conversion chip. Here, N is an integer greater than 1, and the resolution of each sub-image does not exceed the maximum single-channel supported resolution of the FPGA chip.

[0023] Step S20: The signal conversion chip converts the N sub-image signals into N image signals in a format supported by the back-end FPGA chip and transmits them to the FPGA chip.

[0024] In this embodiment, the ARM main control chip may be, for example, a chip with the model number RK3588, the signal conversion chip may be, for example, an HDMI receiver with the model number GSV2015, and the FPGA chip may be a chip with the model number PH1A400SFG900.

[0025] In this embodiment, since the FPGA chip does not support direct input of HDMI signals, the HDMI receiver receives and parses the HDMI signal, converts the received HDMI signal into an RGB TTL signal, and then inputs it into the FPGA chip.

[0026] In step S30, the FPGA chip, based on preset layout information, stitches the received N-channel image signals into a complete image frame with the same resolution as the original image and outputs it to the LED receiver card to drive the screen display.

[0027] This embodiment uses the above technical solution to intelligently split high-resolution images on the front-end ARM chip and accurately stitch them together on the back-end FPGA chip, thereby achieving ultra-high-definition high-resolution display on the LED screen without upgrading the back-end FPGA chip hardware.

[0028] Further, in this embodiment, step S10, where the ARM main control chip splits the high-resolution original image to be displayed into N sub-image signals based on the target display resolution and the maximum single-channel supported resolution of the back-end FPGA chip, includes: The ARM main control chip dynamically selects the splitting direction based on the aspect ratio of the original image. When the width is greater than the height, it splits horizontally; when the height is greater than the width, it splits vertically.

[0029] Furthermore, in this embodiment, before step S30, the method further includes: the ARM main control chip and the back-end FPGA chip transmitting the layout information through a communication interface, wherein the layout information includes at least the starting coordinates, width, and height of each sub-image in the complete image frame. The communication interface may be an SPI serial peripheral interface.

[0030] Furthermore, in this embodiment, the FPGA chip is internally configured with a canvas synthesizer for configuring the display area of ​​each frame.

[0031] Specifically, the canvas synthesizer is configured to receive multiple image inputs and configure independent display layer and display area parameters for each input to synthesize the final image on an internal virtual canvas.

[0032] Furthermore, in this embodiment, the ARM main control chip is configured with multiple independent video output ports, and each video output port is configured to read different sub-region data from the frame buffer of the original image.

[0033] The following example, taking a target display resolution of 7680×2160 and a maximum single-channel input resolution of 3840×2160 supported by the FPGA chip as an example, further illustrates the overall process of this invention.

[0034] Step 1, System Initialization and Target Setting: The system starts up, and the target display resolution is set to 7680×2160. It is known that the maximum supported input resolution for a single channel of the backend FPGA chip is 3840×2160@60Hz. Step 2: Intelligent Splitting at the ARM Main Control Chip: The ARM main control chip detects that the target resolution width (7680) is greater than the height (2160) and automatically adopts a horizontal splitting strategy. The ARM main control chip RK3588 has multiple independent video processing units (VPUs) and display interfaces (Video Ports). The system driver layer configures Video Port 0 to read a 3840×2160 pixel area (i.e., the left half of the image) starting from the top left corner coordinate (0, 0) of the framebuffer; simultaneously, it configures Video Port 1 to read another 3840×2160 pixel area (i.e., the right half of the image) starting from coordinate (3840, 0). The two signals are output through HDMI TX0 and HDMI TX1 interfaces respectively.

[0035] It should be noted that the ARM main control chip automatically splits the screen according to the target display size. For example, when the width is greater than the height, it automatically splits into left and right screens; when the height is greater than the width, it automatically splits into top and bottom screens. Taking the RK3588 chip as an example, the RK3588 chip has 4 independent Video Ports, supporting 4 independent display outputs under basic conditions. Each Video Port can be configured to capture a specified area of ​​the FrameBuffer for display. Using Video Port 0 to capture the left half of the FrameBuffer and Video Port 1 to capture the right half of the FrameBuffer, the screen is split horizontally.

[0036] Step 3, Signal Conversion: The two HDMI signals are respectively connected to two HDMI receiver chips (such as GSV2015). The GSV2015 chip is responsible for decoding the HDMI signal and converting it into two parallel RGB TTL signals, which can be directly received by the FPGA's I / O ports.

[0037] Step 4, SPI Communication Configuration: The ARM master control chip sends a configuration data packet to the FPGA chip via the SPI bus. The data packet contains layout parameters for two sub-screens: First screen, screen ID=1, starting coordinates x=0, y=0, width=3840, height=2160; Second screen, screen ID=2, starting coordinates x=3840, y=0, width=3840, height=2160. The FPGA chip's firmware parses these parameters and stores them in its internal registers.

[0038] Step 5: FPGA Chip Composite: The FPGA chip's two MIPI or parallel video input interfaces receive two RGB TTL signals respectively. An internal canvas compositor IP core is pre-instantiated within the FPGA chip. This IP core internally creates a virtual canvas of size 7680×2160. Based on the parameters from the SPI, it maps the first input signal to layer 1 of the canvas, setting the display area to (0, 0) to (3840, 2160); and maps the second input signal to layer 2 of the canvas, setting the display area to (3840, 0) to (7680, 2160). Within each clock cycle, the canvas compositor automatically selects the pixel data of the corresponding layer based on the pixel coordinates, and composites them in real time into a complete 7680×2160 image stream.

[0039] It should be noted that the FPGA chip can implement a canvas compositor internally. For example, the PH1A400SFG900 chip supports 4-screen composition, and the display area (x, y, width, height) of each screen can be configured separately. If the display area of ​​the first screen is configured as (0, 0, 3840, 2160) and the display area of ​​the second screen is configured as (3840, 0, 3840, 2160), then the total screen size is 7680x2160.

[0040] Step 6, final output display: The synthesized high-definition video stream is transmitted to the LED receiving card through the FPGA's gigabit Ethernet port according to the agreed protocol (such as the manufacturer's custom protocol), and finally drives the LED screen to display a seamless 7680×2160 ultra-wide screen.

[0041] The beneficial effects of the image processing method of the present invention are: 1. Breaking through hardware bottlenecks: The method of splitting and splicing LEDs to achieve high-resolution display is creatively achieved, making full use of chip capabilities and effectively avoiding the single-channel resolution limitation of the back-end FPGA chip. 2. Significantly reduced costs: Without replacing existing FPGA chips or other core hardware, the system performance can be significantly improved through algorithm upgrades, resulting in extremely high economic efficiency.

[0042] 3. Intelligent and flexible adaptation: The image splitting strategy can be automatically adjusted according to the aspect ratio of the displayed content, enhancing the system's adaptability to different application scenarios (such as ultra-wide screens and portrait screens).

[0043] 4. Stable and reliable system: Based on mature SPI communication and FPGA image synthesis technology, the solution is highly stable and easy to implement and promote.

[0044] To achieve the above objectives, the present invention also proposes an image processing system, which includes an ARM main control chip, a signal conversion chip, and an FPGA chip, such as... Figure 3As shown, the system also includes a processor 1001, a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002, and an image processing program stored on the processor. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0045] Those skilled in the art will understand that Figure 3 The system structure shown does not constitute a limitation on the system and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0046] like Figure 3 As shown, the memory 1005, which serves as a computer storage medium, may include an operating device, a network communication module, a user interface module, and an image processing program.

[0047] exist Figure 3 In the system shown, the network interface 1004 is mainly used to connect to the network server and communicate with the network server; the user interface 1003 is mainly used to interact with the user terminal and receive user input instructions; and the processor 1001 can be used to call the image processing program stored in the memory 1005.

[0048] To achieve the above objectives, the present invention also proposes a computer-readable storage medium storing an image processing program, which, when run by a processor, executes the steps of the method described above, which will not be repeated here.

[0049] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that, The method is applied to an image processing system, which includes an ARM main control chip, a signal conversion chip, and an FPGA chip; the method includes the following steps: Step S10: The ARM main control chip splits the high-resolution original image to be displayed into N sub-image signals according to the target display resolution and the maximum single-channel supported resolution of the back-end FPGA chip, and transmits them to the signal conversion chip. Here, N is an integer greater than 1, and the resolution of each sub-image does not exceed the maximum single-channel supported resolution of the FPGA chip. Step S20: The signal conversion chip converts the N sub-image signals into N image signals in a format supported by the back-end FPGA chip and transmits them to the FPGA chip. In step S30, the FPGA chip, based on preset layout information, stitches together the received N-channel image signals into a complete image frame with the same resolution as the original image and then outputs it.

2. The image processing method according to claim 1, characterized in that, In step S10, the step of the ARM main control chip splitting the high-resolution original image to be displayed into N sub-image signals according to the target display resolution and the maximum single-channel supported resolution of the back-end FPGA chip includes: The ARM main control chip dynamically selects the splitting direction based on the aspect ratio of the original image. When the width is greater than the height, it splits horizontally; when the height is greater than the width, it splits vertically.

3. The image processing method according to claim 2, characterized in that, Step S20 includes: The signal conversion chip converts the HDMI signal output by the ARM main control chip into a parallel signal in RGB TTL format.

4. The image processing method according to claim 1, characterized in that, Before step S30, the method further includes: the ARM main control chip and the back-end FPGA chip transmit the layout information through a communication interface, and the layout information includes at least the starting coordinates, width and height of each sub-image in the complete image frame.

5. The image processing method according to claim 4, characterized in that, The communication interface is an SPI serial peripheral interface.

6. The image processing method according to claim 1, characterized in that, The FPGA chip is equipped with a canvas synthesizer, which is used to configure the display area of ​​each image separately.

7. The image processing method according to claim 6, characterized in that, The canvas synthesizer is configured to receive multiple image inputs and configure independent display layer and display area parameters for each input to synthesize the final image on an internal virtual canvas.

8. The image processing method according to claim 1, characterized in that, The ARM main control chip is equipped with multiple independent video output ports, each of which is configured to read different sub-region data from the frame buffer of the original image.

9. An image processing system, characterized in that, The system includes an ARM main control chip, a signal conversion chip, an FPGA chip, a memory, and a processor. The processor stores an image processing program, which, when run by the processor, executes the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image processing program, which, when executed by a processor, performs the steps of the method as described in any one of claims 1 to 8.

Images