Image processing method, device and system

Abstract

The invention provides an image processing method, device and system. The method is applied to the image processing equipment, the image processing equipment comprises a first processor and a second processor, and the first processor is provided with a display interface; the method comprises the steps that a first processor writes a first image to be displayed into a frame buffer area, the resolution of the frame buffer area is matched with the resolution of the first image, the horizontal resolution of the first image is larger than the maximum horizontal resolution supported by a display interface, and / or the vertical resolution of the first image is larger than the maximum vertical resolution supported by the display interface; the first processor controls the display interface to output a second image to the second processor according to the first image in the frame buffer area, the horizontal resolution of the second image is smaller than or equal to the maximum horizontal resolution supported by the display interface, and the vertical resolution of the second image is smaller than or equal to the maximum vertical resolution supported by the display interface; the second processor transmits image data to the display device according to the second image, so that the display device displays the first image. According to the technical scheme, the image processing efficiency of the image processing equipment can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of display screens, and more particularly to an image processing method, device and system. BACKGROUND

[0002] The image processing device can process (such as decode and render) an image issued by an image source based on a first processor (such as an ARM processor). After processing the image, the first processor can first store the processed image into a frame buffer, and then send the image cached in the frame buffer to a second processor (such as an FPGA) by using a display interface, so as to drive a display device to display the image by the second processor.

[0003] The output resolution of the display interface of the first processor is physically limited. If the resolution of the image to be displayed exceeds the physical limit, the first processor needs to split the image and then store the split image into the frame buffer. The image splitting operation will occupy the processing resources of the image processing device, thereby reducing the processing efficiency of the image processing device. SUMMARY

[0004] Embodiments of the present application provide an image processing method, device and system to improve the image processing efficiency of the image processing device. Each aspect of the present application is introduced as follows.

[0005] In a first aspect, an image processing method is provided. The method is applied to an image processing device, the image processing device comprising a first processor and a second processor, the first processor having a display interface. The method comprises: the first processor writing a first image to be displayed into a frame buffer, a resolution of the frame buffer matching a resolution of the first image, a horizontal resolution of the first image being greater than a maximum horizontal resolution supported by the display interface, and / or a vertical resolution of the first image being greater than a maximum vertical resolution supported by the display interface; the first processor controlling the display interface to output a second image to the second processor according to the first image in the frame buffer, a horizontal resolution of the second image being less than or equal to the maximum horizontal resolution supported by the display interface, and a vertical resolution of the second image being less than or equal to the maximum vertical resolution supported by the display interface; and the second processor transmitting image data to a display device according to the second image, so that the display device displays the first image.

[0006] In some implementations, the second processor transmitting image data to a display device according to the second image comprises: the second processor converting the second image into the first image; and the second processor transmitting image data corresponding to the first image to the display device.

[0007] In some embodiments, the first image comprises a plurality of image blocks arranged in a first direction in the first image, and the second image comprises the plurality of image blocks arranged in a second direction in the second image, wherein the first direction is a horizontal direction and the second direction is a vertical direction, or the first direction is a vertical direction and the second direction is a horizontal direction.

[0008] In some embodiments, the first processor comprises a display controller, and the first processor controls the display interface to output a second image to the second processor according to the first image in the frame buffer, comprising: the first processor acquires the plurality of image blocks from the frame buffer by using the display controller; and the first processor outputs the plurality of image blocks to the display interface in sequence by using the display controller.

[0009] In some embodiments, a row of pixels in the first image comprises a plurality of groups of pixels, each group of pixels comprising a plurality of consecutive pixels in the row of pixels, the second image comprises the plurality of groups of pixels, and the plurality of groups of pixels form a plurality of adjacent rows of pixels in the second image, or a column of pixels in the first image comprises a plurality of groups of pixels, each group of pixels comprising a plurality of consecutive pixels in the column of pixels, the second image comprises the plurality of groups of pixels, and the plurality of groups of pixels form a plurality of adjacent columns of pixels in the second image.

[0010] In some embodiments, the first processor comprises a display controller, and the first processor controls the display interface to output a second image to the second processor according to the first image in the frame buffer, comprising: the first processor acquires the plurality of groups of pixels from the frame buffer by using the display controller; and the first processor outputs the plurality of groups of pixels to the display interface in sequence by using the display controller.

[0011] In some embodiments, the first processor is an ARM processor, and the second processor is an FPGA.

[0012] In a second aspect, an image processing device is provided, which includes a first processor and a second processor, the first processor having a display interface; the first processor is configured to: write a first image to be displayed into a frame buffer, a resolution of the frame buffer matching a resolution of the first image, a horizontal resolution of the first image being greater than a maximum horizontal resolution supported by the display interface, and / or a vertical resolution of the first image being greater than a maximum vertical resolution supported by the display interface; the first processor is further configured to: control the display interface to output a second image to the second processor according to the first image in the frame buffer, a horizontal resolution of the second image being less than or equal to the maximum horizontal resolution supported by the display interface, and a vertical resolution of the second image being less than or equal to the maximum vertical resolution supported by the display interface; the second processor is configured to: transmit image data corresponding to the first image to a display device, so that the display device displays the first image.

[0013] In some implementations, the second processor is configured to: the second processor converts the second image into the first image; and the second processor transmits image data corresponding to the first image to the display device.

[0014] In some implementations, the first image includes a plurality of image blocks, the plurality of image blocks being arranged in a first direction in the first image; the second image includes the plurality of image blocks, and the plurality of image blocks being arranged in a second direction in the second image; wherein the first direction is a horizontal direction, and the second direction is a vertical direction; or the first direction is a vertical direction, and the second direction is a horizontal direction.

[0015] In some implementations, the first processor includes a display controller, and the first processor is further configured to: acquire the plurality of image blocks from the frame buffer by using the display controller; and output the plurality of image blocks to the display interface in sequence by using the display controller.

[0016] In some implementations, a row of pixels in the first image includes a plurality of groups of pixels, each group of pixels including a plurality of consecutive pixels in the row of pixels, the second image includes the plurality of groups of pixels, and the plurality of groups of pixels form a plurality of adjacent rows of pixels in the second image; or a column of pixels in the first image includes a plurality of groups of pixels, each group of pixels including a plurality of consecutive pixels in the column of pixels, the second image includes the plurality of groups of pixels, and the plurality of groups of pixels form a plurality of adjacent columns of pixels in the second image.

[0017] In some implementations, the first processor comprises a display controller, and the first processor is further configured to: acquire the plurality of groups of pixels from the frame buffer by using the display controller; and output the plurality of groups of pixels to the display interface in sequence by using the display controller.

[0018] In some implementations, the first processor is an ARM processor, and the second processor is an FPGA.

[0019] In a third aspect, an image processing system is provided, comprising: the image processing device according to the second aspect or any one of the second aspects; and a display device connected with the image processing device.

[0020] In some implementations, the display device comprises a plurality of LED light boxes, and a resolution of a display screen formed by the plurality of LED light boxes matches a resolution of the first image. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1A A system architecture to which embodiments of the present application can be applied is shown.

[0022] Figure 1B A possible implementation of the image processing system in Figure 1A is shown.

[0023] Figure 2 Another example of an image processing process provided by embodiments of the present application is shown.

[0024] Figure 3 A schematic flowchart of an image processing method provided by embodiments of the present application is shown.

[0025] Figure 4 An example of an image resolution conversion manner provided by embodiments of the present application is shown.

[0026] Figure 5 Another example of an image processing process provided by embodiments of the present application is shown.

[0027] Figure 6 Still another example of an image processing process provided by embodiments of the present application is shown. DETAILED DESCRIPTION

[0028] Referring to Figure 1A , the image processing system 1 provided by embodiments of the present application comprises an image processing device 10 and a display device 20. The display device 20 can be an LED display device. The LED display device comprises an LED display screen, which can be a common LED display screen, a microLED display screen or a miniLED display screen, or a new type of LED display screen in the future.

[0029] The image processing device 10 can communicate with the host computer 2, so as to receive an image to be displayed issued by the host computer 2. The host computer 2 mentioned herein can be any type of electronic device capable of communicating with the image processing device 10, such as a PC computer, a mobile terminal, or a cloud server, etc. After receiving the image, the image processing device 10 can process (such as decode and / or render) the image, and then the image processing device 10 can send image data to the display device 20, so that the display device 20 displays the image.

[0030] The image processing device 10 can include a first processor 12 and a second processor 14 which cooperate with each other. In some embodiments, the first processor 12 can serve as a master processor (or master chip) in the image processing device 10, and the second processor 14 can serve as a co-processor in the image processing device 10. Exemplarily, the first processor 12 can be a general-purpose processor (such as an ARM processor), and the second processor 14 can be an FPGA. The ARM processor has the characteristics of strong computing power, low power consumption, and high integration, and the FPGA has strong parallel processing capability. The two can cooperate with each other to achieve efficient processing of image data.

[0031] The first processor 12 can run a player and use the player to decode and render the image to be displayed, and store the rendered image into a frame buffer. The frame buffer can also be referred to as a display memory, which can be a storage area in the memory of the first processor 12 dedicated for storing the image to be displayed. In addition to the frame buffer, the first processor 12 also has a display interface 122. The first processor 12 can be connected (communicatively connected) with the second processor 14 through the display interface 122, so as to output the image to the second processor 14. The type of the display interface is not specifically limited in the embodiments of the present application. For example, the display interface can be HDMI or LVDS.

[0032] The second processor 14 can be responsible for transmitting the image to the display device 20, so that the display device 20 can display the image. Taking an LED display screen as an example, the second processor 14 can be connected with an LED receiving card on the LED display screen through a network port or an optical fiber interface. After obtaining the image, the second processor 14 can send the image to the LED receiving card through the network port or the optical fiber interface, so as to display the image on the LED display screen.

[0033] For ease of understanding, the implementation of the image processing system 1 shown in Figure 1B will be described in more detail below. Figure 1A

[0034] Reference is made to Figure 1B ​, the image processing device 10 can include an ARM processor and an FPGA, wherein the ARM processor corresponds to the first processor mentioned above, and the FPGA corresponds to the second processor mentioned above. The ARM processor can include a CPU, a GPU, a memory, an LCD display controller (LCDC) and the like. Among them, the CPU can include an application layer, an operating system and a driver layer. The application layer can run application software such as a player, for example. The driver layer is a bridge between the operating system and other hardware in the ARM processor. The driver layer can run the driver program of other hardware in the ARM processor. For example, the ARM processor includes an LCD display controller, which can control the display interface to output image data to the FPGA. The LCD display controller is independent hardware in the ARM processor, and the CPU in the ARM processor can control the control behavior of the LCD display controller through the driver program in the driver layer. The ARM processor further includes a memory. The CPU can create a frame buffer in the memory to cache image data.

[0035] The FPGA can receive the image data output by the LCD display controller through the display interface, and transmit the image data to the receiving card of the LED display device (for example, an LED display device) through a network port or a fiber interface. After receiving the image data, the receiving card can control the LED display screen to display images. Figure 1B

[0036] With the development of display technology, in many occasions, it is necessary to display ultra-wide images (or display ultra-high images by using ultra-high display devices, hereinafter mainly taking the display of ultra-wide images as an example) by using ultra-wide display devices, such as an image with a resolution of 10000x256. For example, in a stadium, a super-wide display screen can be used to live broadcast the game picture, play the highlight, display advertisements and the like. For another example, in a shopping mall or exhibition hall, a super-wide display screen can be used to display rich product information, advertisement content and the like to attract the attention of consumers.

[0037] However, as mentioned above, the resolution supported by the display interface 122 of the first processor 12 (or the output resolution of the display interface 122) is usually physically limited. Taking an ARM processor as an example, the maximum resolution supported by the display interface of the ARM processor can be 4096x2160. That is, if the horizontal resolution of the original image is greater than 4096 and / or the vertical resolution is greater than 2160, the original image cannot be directly output through the display interface.

[0038] ​In order to enable the display device to display the ultra-wide image, one possible implementation is to split the ultra-wide image into a plurality of image blocks by using the application software running on the first processor 12, and then stitch the plurality of image blocks so that the resolution of the stitched image meets the resolution requirement of the display interface 122. Then, the display interface 122 can be controlled to output the stitched image to the second processor 14. After receiving the stitched image, the second processor 14 can send the stitched image to the display device for display.

[0039] The above implementation will be described in more detail below with reference to the example of displaying an image with a resolution of 10000x256 on an LED display device. Figure 2 In the example of the above implementation, Figure 2 In the example of the above implementation, the first processor mentioned in the foregoing is an ARM processor, and the second processor is an FPGA processor.

[0040] Referring to Figure 2 The host computer software sends an ultra-wide image A with a resolution of 10000x256 to the image processing device. The ARM processor in the image processing device first decodes the image A by using the application software. Assuming that the width and height limit of the display interface of the ARM processor is 2500x1024, the image A cannot be directly output through the display interface. In this case, the ARM processor can split the image A into four image blocks B by using the application software, and each image block B has a resolution of 2500x256. Then, the ARM processor can render the four image blocks B to different areas in the frame buffer respectively to realize the stitching of the four image blocks B, thereby obtaining a stitched image C. From Figure 2 It can be seen that the resolution of the image C is 2500x1024, which meets the resolution requirement of the display interface of the ARM processor. Then, the ARM processor can output the image C to the FPGA through the display interface. The FPGA can send the image data in the image C directly to the LED receiving card of the LED display device through the network port or the optical fiber interface, and finally present the ultra-wide image A with a resolution of 10000x256 on the LED display device. It should be noted that since the resolution of the image C is 2500x1024, and the resolution of the ultra-wide image A finally presented on the LED display device is 10000x256, the user needs to configure the resolution of the LED display device to 2500x1024 by using the screen matching software, and manually move the LED box to form the shape of the ultra-wide screen.

[0041] The implementation manner above can enable the display device to display the ultra-wide image, but has some defects. For example, the implementation manner above needs the first processor to perform segmentation on the ultra-wide image by using the application software after decoding the ultra-wide image, and to re-render the segmented image to the corresponding storage location of the frame buffer, which has high software implementation complexity. In addition, these additional introduced image segmentation and other operations are implemented at the application layer, which can greatly consume the processing resources (such as CPU and / or GPU resources) on the first processor, thereby reducing the image processing efficiency of the image processing device, and further can cause the image playing process to have a freezing phenomenon. It is also worth mentioning that the implementation manner above also has poor compatibility, that is, not all images can support the image segmentation operation. For example, seamless spliced video images, full-screen overlaid static pictures, dynamic special effects, web page images, H5 programs, and application software interfaces and other image contents often cannot or are difficult to be segmented, and if forced to be segmented, display abnormalities can occur.

[0042] To solve the above problems, the following will be combined with Figure 3 to introduce the embodiments of the present application in detail.

[0043] Figure 3 The schematic flowchart of the image processing method provided by the embodiments of the present application is shown in FIG. 3. Figure 3 The method shown in FIG. 3 can be implemented by the image processing device mentioned above. Specifically, Figure 3 The method shown in FIG. 3 can be completed by the first processor and the second processor in the image processing device in cooperation with each other.

[0044] Referring to Figure 3 At step S310, the first processor writes the first image to be displayed into the frame buffer. The horizontal resolution of the first image is greater than the maximum horizontal resolution supported by the display interface of the first processor, and / or the vertical resolution of the first image is greater than the maximum vertical resolution supported by the display interface. That is, the first image cannot be directly output through the display interface of the first processor. As an example, the first image is an ultra-wide image with a resolution of 10000x256.

[0045] In the related art, the resolution of the frame buffer of the first processor and the output resolution of the display interface of the first processor are usually set to the same resolution. For example, if the output resolution of the display interface of the first processor is set to 4000x768, the first processor will create a frame buffer with a resolution of 4000x768 in the memory. In this way, the first processor can directly output the image cached in the frame buffer to the outside through the display interface.

[0046] However, if the above-mentioned processing manner of the related art is followed, the first image cannot be directly cached into the frame buffer, because the horizontal and / or vertical resolution of the first image to be displayed exceeds the maximum horizontal and / or vertical resolution that the display interface can support. It is for this reason, Figure 2 The implementation shown requires that the first image be segmented first, and then each segmented image block is rendered to the correct position of the frame buffer. However, this requires the introduction of additional segmentation and rendering operations, which will occupy a large amount of processing resources of the first processor.

[0047] Unlike the related art, the embodiments of the present application no longer require that the resolution of the frame buffer be consistent with the output resolution of the display interface, but instead require that the resolution of the frame buffer match the resolution of the first image. The "match" mentioned here means that the resolution of the frame buffer is set such that the frame buffer can directly accommodate the complete first image without the need for segmentation of the first image. For example, the horizontal resolution of the frame buffer is greater than or equal to the horizontal resolution of the first image, and the vertical resolution of the frame buffer is greater than or equal to the vertical resolution of the first image. Exemplarily, the resolution of the frame buffer is the same as the resolution of the first image.

[0048] Continuing to refer to Figure 3 At step S320, the first processor controls the display interface to output a second image to the second processor according to the first image in the frame buffer. The horizontal resolution of the second image is less than or equal to the maximum horizontal resolution supported by the display interface, and the vertical resolution of the second image is less than or equal to the maximum vertical resolution supported by the display interface. In other words, the second image is adapted to the output resolution of the display interface and can be directly output through the display interface.

[0049] Step S320 describes the output control process of the display interface. It can be understood that the control of the display interface can be taken care of by a display controller (such as an LCD display controller) in the first processor. The display controller is usually a separate hardware module within the first processor. Figure 1B

[0050] ​In the related art, because the resolution of the frame buffer is the same as the output resolution of the display interface, the display controller controls the output manner of the image in the frame buffer, so that the pixels of the image in the frame buffer are output to the display interface row by row. Unlike the related art, in the embodiments of the present application, the frame buffer stores a first image with a larger resolution (exceeding the resolution supported by the display interface), so the pixels in the frame buffer cannot be output to the display interface row by row as in the related art, but the output control manner of the pixels in the frame buffer needs to be adjusted, so that the resolution of the second image output by the display interface meets the limitation requirement of the output resolution of the display interface. For example, if the horizontal resolution of the first image exceeds the maximum horizontal resolution supported by the display interface, the row of pixels can be segmented and output to the display interface, so that the row of pixels in the first image forms multiple rows of pixels in the second image. Of course, other output control manners can also be used to output the pixels in the frame buffer to the display interface, as long as the pixels transported to the display interface at each time do not exceed the resolution limitation of the display interface. For details, please refer to Embodiment 1 and Embodiment 2 in the following, which will not be described here.

[0051] As mentioned above, the display controller of the first processor is usually an independent hardware module, and the adjustment of the control behavior of the hardware module can be realized by configuring the registers of the hardware module in the driver layer (which means that the display controller can be controlled to perform similar operations as described above by configuring the registers according to the conversion logic in the driver layer). Figure 2 As can be seen, the control behavior described in step S320 can be realized by the driver layer and the hardware layer of the first processor, and the whole process does not need excessive participation of the CPU and / or GPU in the first processor, so the processing resources of the first processor can be saved, and the image processing efficiency of the image processing device can be improved.

[0052] Continuing to refer to Figure 3 In step S330, the second processor transmits image data to the display device according to the second image, so that the display device displays the first image.

[0053] The embodiment of the present application does not make a specific limitation on the implementation of step S330. In some implementations, the second processor can directly transmit the second image to the display device, so that the display device displays the first image. For example, the user can configure the resolution of the display device to be the resolution of the second image through the screen matching software, and then the user can manually move the display modules (such as LED boxes) in the display device, so that the display screen formed by the plurality of display modules has the same resolution as the first image. In this way, the second processor can directly send the image data corresponding to the second image, and since the display modules in the display device are arranged according to the resolution of the first image, the first image will be displayed on the display screen of the display device.

[0054] If the above implementation is adopted, the second processor only needs to act as a repeater to transmit the image data corresponding to the second image to the display device for display, so that the implementation of the second processor can be simplified. However, the above implementation requires the user to understand the mapping relationship between the resolution configured by the screen matching software and the real physical layout of the display modules in the display device, which increases the difficulty of learning and operation for the user.

[0055] Therefore, in some other implementations, the second processor can first convert the second image into the first image, and then transmit the image data corresponding to the first image to the display device. In this way, when the user uses the screen matching software to match the screen, the user can directly match the screen according to the real physical layout of the display device, so as to reduce the complexity of learning and operation for the user.

[0056] The foregoing step S320 mentions that the first processor controls the display interface to output the second image to the second processor according to the first image in the frame buffer. There are many implementation manners of step S320, and two possible implementation manners are given in combination with Embodiment One and Embodiment Two.

[0057] Example One In Embodiment One, the first image includes a plurality of image blocks, and the plurality of image blocks are arranged in the first image along a first direction; the second image also includes the plurality of image blocks, and the plurality of image blocks are arranged in the second image along a second direction.

[0058] In some embodiments, the first direction mentioned above can be a horizontal direction, and the second direction can be a vertical direction. That is, based on step S320, the first image is equivalent to being segmented, and the plurality of image blocks obtained after the segmentation are re-pasted, so that the first image with a large width-to-height ratio becomes the second image with a small width-to-height ratio.

[0059] In other embodiments, the first direction mentioned above can be a vertical direction, and the second direction can be a horizontal direction. That is, based on step S320, it is equivalent to performing a segmentation operation on the first image and re-stitching the multiple image blocks obtained after segmentation, so that the first image with a large aspect ratio becomes the second image with a small aspect ratio.

[0060] The conversion process from the first image to the second image described above can be achieved by controlling the order in which multiple image blocks are output to the display interface. For example, the first processor can use the display controller to obtain multiple image blocks from the frame buffer; then, the first processor can use the display controller to output these multiple image blocks sequentially to the display interface. That is, the first processor uses the display controller to first output one image block from the multiple image blocks to the display interface, and then outputs the next image block to the display interface, and so on, until all image blocks have been output to the display interface. When outputting each image block to the display interface, the first processor can use the display controller to output the pixels in each image block line by line to the display interface.

[0061] by Figure 4 For example, the first image includes a first image block, a second image block, and a third image block, arranged horizontally. The first processor can use the display controller to first output the first image block to the display interface, then the second image block, and finally the third image block. After outputting the three image blocks to the display interface in this order, the output image of the display interface is the second image. Therefore, the process of using the display controller to send the various image blocks is equivalent to implementing the segmentation and splicing operation of the first image block, thus completing the conversion from the first image to the second image. The control behavior of this display controller can be implemented by configuring the display controller's registers in the driver layer (meaning that the display controller can perform the conversion from the first image to the second image by controlling the registers according to the conversion logic in the driver layer), without occupying the CPU and / or GPU resources of the first processor.

[0062] During the process of receiving multiple image blocks output from the display interface, the second processor can store these multiple image blocks in an image buffer in a horizontal arrangement, thereby restoring the second image into the first image. In other words, the first processor uses the display controller to fold the first image in the column direction, and the second processor restores the first image in the row direction. The entire process does not require the application layer of the first processor to participate.

[0063] To facilitate understanding, the following will be combined with... Figure 5The following example illustrates Embodiment 1 in more detail. In the example below, the first processor is an ARM processor, the second processor is an FPGA, the display device is an LED display device, and the display screen of the LED display device is an ultra-wide display screen with a resolution of 12000×256. The resolution of the first image is 12000×256, the resolution of the second image is 4000x768, and the width and height of the resolution supported by the display interface of the ARM processor are both limited to 4096.

[0064] First, the ARM processor creates a frame buffer. This frame buffer has a resolution of 12000×256. In other words, the ARM processor allocates buffer space for the frame buffer according to a resolution of 12000×256.

[0065] Next, the ARM processor configures the parameters of the display interface. Specifically, the ARM processor can configure the output resolution of the display interface to 4000x768. In addition, the ARM processor can further set the timing parameters of the display interface (such as horizontal sync signals, vertical sync signals, and other related parameters) and initialize the registers of the display controller.

[0066] Next, the playback software (application layer software) in the ARM processor decodes the first image and writes the decoded first image directly into the frame buffer. Since the resolution of the frame buffer matches the resolution of the first image, the frame buffer can directly hold the first image.

[0067] Then, the ARM processor can divide the frame buffer (or the first image stored in the frame buffer) into three image blocks, namely... Figure 5 The image shown consists of a first image block, a second image block, and a third image block. Each of these three image blocks has a resolution of 4000×256. Furthermore, the AMR processor can create an index table for the image blocks and record the address and size of each image block.

[0068] Next, the display controller in the ARM processor can control the output of three image blocks. For example, in the first time period, the ARM processor outputs the first image block line by line to the display interface; in the second time period, the ARM processor outputs the second image block line by line to the display interface; and in the third time period, the ARM processor outputs the third image block line by line to the display interface. In this way, the display interface outputs as follows: Figure 5 The second image shown. From Figure 5 As can be seen, the resolution of the second image is 4000x768, which meets the output resolution limit requirements of the display interface.

[0069] During the process of sequentially outputting three image blocks from the display interface, the FPGA can store the three image blocks in their corresponding positions in the image buffer according to their arrangement in the first image, thereby restoring the second image to the first image. Then, the FPGA can transmit the image data corresponding to the first image to the receiving card of the LED display device, enabling the LED display device to display the first image.

[0070] As can be seen from the above description, the solution provided in this application embodiment can drive images far exceeding the output capability limits of an ARM processor with limited display interface output capabilities. Furthermore, the ARM processor's control over the output process from pixels in the first image to the display interface does not require application layer involvement (i.e., it does not require...). Figure 2 This approach, where the first image is segmented at the application layer, reduces software development and maintenance costs. Furthermore, since image segmentation is not required at the application layer, it avoids consuming processing resources of the primary processor (such as CPU and / or GPU resources), resulting in smoother operation of the entire image processing system.

[0071] Example Two In Embodiment 2, a row of pixels in the first image comprises multiple groups of pixels, where each group of pixels includes a plurality of consecutive pixels within a row. The second image also includes these multiple groups of pixels, and these multiple groups of pixels form adjacent rows of pixels in the second image. That is, based on step S320, a row of pixels in the first image is effectively segmented, causing a row of pixels in the first image to form adjacent rows of pixels in the second image. If each row of pixels in the first image is transformed in this manner, the first image with a larger aspect ratio will become the second image with a smaller aspect ratio.

[0072] The first controller can use the display controller to convert a row of pixels in the first image into multiple rows of pixels in the second image. For example, the first controller can use the display controller to first output a group of pixels in a row to the display interface. After that group of pixels has been output, the first controller can then use the display controller to output the next group of pixels in the same row to the display interface, and so on. After processing a row of pixels, the first controller can process the next row of pixels in the first image in a similar manner, until all pixels of the first image have been output to the display interface.

[0073] It can be understood that the pixels in the first image need to be copied before being output to the display interface. The advantage of the second implementation manner is that since the storage addresses of a row of pixels in the frame buffer are usually continuous, the display controller of the first processor can copy the whole row of pixels from the frame buffer at one time and then output the multiple groups of pixels to the display interface respectively. That is, the display controller of the first processor can complete the output of the multiple groups of pixels by using one copy operation, and does not need to perform a copy operation before outputting the pixels to the display interface each time, thereby simplifying the operation complexity of the first processor (or the display controller in the first processor) and improving the image processing efficiency.

[0074] In the process of receiving the multiple image blocks output by the display interface, the second processor can re-splice the multiple rows of adjacent pixels in the second image into one row of pixels, so as to restore the second image to the first image. That is, the first processor folds the first image in the column direction by using the display controller, and the second processor restores the first image in the row direction, and the whole process does not need the application layer of the first processor to participate.

[0075] For the convenience of understanding, the following will be described in combination with Figure 6 The second embodiment will be described in more detail. In the following examples, the first processor is an ARM processor, the second processor is an FPGA, the display device is an LED display device, the display screen of the LED display device is an ultra-wide display screen, the resolution of the LED display device is 10000x256, the resolution of the first image is 10000x256, the resolution of the second image is 2500x1024, and the output resolution limit of the display interface of the ARM processor is 2500x1024.

[0076] First, the ARM processor opens a frame buffer with the same resolution as the LED display device in the memory. Specifically, since the resolution of the LED display device is 10000x256, a 10000x256 frame buffer is directly created in the memory of the ARM processor. When the application layer (including the business software and / or the player) of the ARM processor receives the first image with the resolution of 10000x256 from the host computer, the application layer of the ARM processor decodes and renders the first image, and stores the first image after decoding and rendering into the frame buffer.

[0077] Then, the display controller of the ARM processor outputs the first image (with the resolution of 10000x256) to the display interface, and controls the output process of the pixels in the first image, so that the display interface outputs the second image with the resolution of 2500x1024.

[0078] For example, the ARM processor can cut and rearrange each row of pixels (10,000 pixels) of the first image in sequence into 4 rows of pixels (2,500 pixels per row). The 1st-2,500th pixels of the 1st row of the first image (corresponding to "1" in Figure 6 ) constitute the 1st row of pixels of the second image, the 2,501st-5,000th pixels of the 1st row of the first image constitute the 2nd row of pixels of the second image (corresponding to "2" in Figure 6 ), the 5,001st-7,500th pixels of the 1st row of the first image constitute the 3rd row of pixels of the second image (corresponding to "3" in Figure 6 ), and the 7,501st-10,000th pixels of the 1st row of the first image constitute the 4th row of pixels of the second image (corresponding to "4" in Figure 6 ). After the 1st row of pixels of the first image is processed, the 2nd row of pixels of the first image can be processed in sequence, for example, the 1st-2,500th pixels of the 2nd row of the first image constitute the 5th row of pixels of the second image, the 2,501st-5,000th pixels of the 2nd row of the first image constitute the 6th row of the second image, the 5,001st-7,500th pixels of the 2nd row of the first image constitute the 7th row of pixels of the second image, and the 7,501st-10,000th pixels of the 2nd row of the first image constitute the 8th row of pixels of the second image, and so on. Through the above process, the "extremely wide" first image can be converted into a "relatively high" second image, so as to adapt to the highest output resolution (2,500x1,024) of the display interface. Moreover, the above process can be completed by the display controller in the ARM processor by using a driver, without excessive involvement of the CPU and / or GPU.

[0079] After the ARM processor transmits the second image (resolution: 2,500x1,024) to the FPGA through the display interface (such as LVDS, HDMI), the FPGA performs the inverse operation of the above operation process. For example, the FPGA can re-cache, splice, and restore each 4 rows of pixels (2,500 pixels per row) received into one row of 10,000 pixels. Through the inverse operation of the FPGA, the FPGA can finally reconstruct the complete first image (resolution: 10,000x256) in the cache. Then, the FPGA sends the first image to the LED receiving card through the network or fiber interface, and then drives the LED display device to complete the display.

[0080] Alternatively, in some embodiments, the first image can be an image with a large aspect ratio. In this case, the column of pixels in the first image can include a plurality of groups of pixels, where each group of pixels includes a plurality of consecutive pixels in the column of pixels. The second image also includes the plurality of groups of pixels, and the plurality of groups of pixels form a plurality of adjacent columns of pixels in the second image. That is, based on step S320, a segmentation operation is performed on the column of pixels in the first image, such that the column of pixels in the first image forms the plurality of adjacent columns of pixels in the second image. If each column of pixels in the first image is converted in this manner, the first image with a large aspect ratio is converted into the second image with a small aspect ratio. The process of converting the column of pixels in the first image into the plurality of adjacent columns of pixels in the second image is similar to the process of converting the row of pixels in the first image into the plurality of adjacent rows of pixels in the second image, and details are described above and will not be repeated here.

[0081] It should be noted that the resolution of the frame buffer mentioned in various embodiments of the present application can refer to the width and height of the image stored in the frame buffer.

[0082] It should also be noted that the image mentioned in various embodiments of the present application refers to any type of content that can be displayed or played by a display device, such as a video image, a static image, a dynamic special effect, web content, or a display interface of software, etc. Accordingly, the image processing device 10 mentioned in various embodiments of the present application can also be referred to as a video processor, a playback processor, or a media processor.

[0083] As shown in FIG. 1, the image processing system 1 provided in an embodiment of the present application includes an image processing device 10 and a display device 20. Figure 1A As shown in FIG. 1, the image processing system 1 provided in an embodiment of the present application includes an image processing device 10 and a display device 20.

[0084] It should be understood that, in various embodiments of the present application, the sequence of the above processes does not mean the order of execution, and the execution order of the processes should be determined according to their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0085] In several embodiments provided by the present application, it should be understood that the disclosed system, device, and method can be implemented by other means. For example, the above-described device embodiments are only illustrative, and for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed ones can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0086] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0087] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0088] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can read, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0089] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method is applied to an image processing device, which includes a first processor and a second processor, wherein the first processor has a display interface. The method includes: The first processor writes the first image to be displayed into a frame buffer, the resolution of which matches the resolution of the first image, the horizontal resolution of the first image being greater than the maximum horizontal resolution supported by the display interface, and / or the vertical resolution of the first image being greater than the maximum vertical resolution supported by the display interface. The first processor controls the display interface to output a second image to the second processor based on the first image in the frame buffer. The horizontal resolution of the second image is less than or equal to the maximum horizontal resolution supported by the display interface, and the vertical resolution of the second image is less than or equal to the maximum vertical resolution supported by the display interface. The second processor transmits image data to the display device based on the second image, so that the display device displays the first image.

2. The method according to claim 1, characterized in that, The second processor transmits image data to the display device based on the second image, including: The second processor converts the second image into the first image; The second processor transmits the image data corresponding to the first image to the display device.

3. The method according to claim 1 or 2, characterized in that, The first image includes a plurality of image blocks, which are arranged along a first direction in the first image; The second image includes the plurality of image blocks, and the plurality of image blocks are arranged along a second direction in the second image; Wherein, the first direction is a horizontal direction and the second direction is a vertical direction; or, the first direction is a vertical direction and the second direction is a horizontal direction.

4. The method according to claim 3, characterized in that, The first processor includes a display controller, which controls the display interface to output a second image to the second processor based on the first image in the frame buffer, including: The first processor uses the display controller to obtain the plurality of image blocks from the frame buffer; The first processor uses the display controller to sequentially output the plurality of image blocks to the display interface.

5. The method according to claim 1 or 2, characterized in that, A row of pixels in the first image includes multiple groups of pixels, wherein each group of pixels includes multiple consecutive pixels in the row of pixels; the second image includes the multiple groups of pixels, and the multiple groups of pixels form adjacent rows of pixels in the second image. or, The first image has a column of pixels comprising multiple groups of pixels, wherein each group of pixels comprises multiple consecutive pixels in the column of pixels, and the second image comprises the multiple groups of pixels, wherein the multiple groups of pixels form adjacent columns of pixels in the second image.

6. The method according to claim 5, characterized in that, The first processor includes a display controller, which controls the display interface to output a second image to the second processor based on the first image in the frame buffer, including: The first processor uses the display controller to obtain the multiple sets of pixels from the frame buffer; The first processor uses the display controller to sequentially output the multiple groups of pixels to the display interface.

7. The method according to claim 1 or 2, characterized in that, The first processor is an ARM processor, and the second processor is an FPGA.

8. An image processing device, characterized in that, The image processing device includes a first processor and a second processor, wherein the first processor has a display interface; The first processor is configured to: write a first image to be displayed into a frame buffer, the resolution of the frame buffer being matched with the resolution of the first image, the horizontal resolution of the first image being greater than the maximum horizontal resolution supported by the display interface, and / or, the vertical resolution of the first image being greater than the maximum vertical resolution supported by the display interface; The first processor is further configured to: control the display interface to output a second image to the second processor based on the first image in the frame buffer, wherein the horizontal resolution of the second image is less than or equal to the maximum horizontal resolution supported by the display interface, and the vertical resolution of the second image is less than or equal to the maximum vertical resolution supported by the display interface; The second processor is configured to transmit image data to a display device based on the second image, so that the display device displays the first image.

9. The image processing apparatus according to claim 8, characterized in that, The second processor is further configured as follows: Convert the second image into the first image; The image data corresponding to the first image is transmitted to the display device.

10. The image processing apparatus according to claim 8 or 9, characterized in that, The first image includes a plurality of image blocks, which are arranged along a first direction in the first image; The second image includes the plurality of image blocks, and the plurality of image blocks are arranged along a second direction in the second image; Wherein, the first direction is a horizontal direction and the second direction is a vertical direction; or, the first direction is a vertical direction and the second direction is a horizontal direction.

11. The image processing apparatus according to claim 10, characterized in that, The first processor includes a display controller, and the first processor is further configured to: The display controller is used to obtain the plurality of image blocks from the frame buffer; The display controller is used to sequentially output the plurality of image blocks to the display interface.

12. The image processing apparatus according to claim 8 or 9, characterized in that, A row of pixels in the first image includes multiple groups of pixels, wherein each group of pixels includes multiple consecutive pixels in the row of pixels; the second image includes the multiple groups of pixels, and the multiple groups of pixels form adjacent rows of pixels in the second image. or, The first image has a column of pixels comprising multiple groups of pixels, wherein each group of pixels comprises multiple consecutive pixels in the column of pixels, and the second image comprises the multiple groups of pixels, wherein the multiple groups of pixels form adjacent columns of pixels in the second image.

13. The image processing apparatus according to claim 12, characterized in that, The first processor includes a display controller, and the first processor is further configured to: The display controller is used to obtain the multiple sets of pixels from the frame buffer; The display controller is used to sequentially output the multiple groups of pixels to the display interface.

14. The image processing apparatus according to claim 8 or 9, characterized in that, The first processor is an ARM processor, and the second processor is an FPGA.

15. An image processing system, characterized in that, include: The image processing apparatus as described in any one of claims 8 to 14; as well as A display device connected to the image processing device.

16. The image processing system according to claim 15, characterized in that, The display device includes multiple LED light boxes, and the resolution of the display screen formed by the multiple LED light boxes matches the resolution of the first image.

Images