Image processing display device and method and computer storage medium

Abstract

The invention provides an image processing display device and method. The device comprises a coordination control processor, a graphic hardware acceleration engine, an LCD (Liquid Crystal Display) controller, an SPI (Serial Peripheral Interface) controller, a DMA (Direct Memory Access) controller and a storage unit, every two parts are interconnected through the bus interconnection matrix; the graphic hardware acceleration engine is used for decoding and rendering the fragmented image data stored in the storage unit; the LCD controller is used for sending the processed data to the display end according to the time sequence requirement of the display interface; the coordination control processor is used for being responsible for coordination control of each module in the whole image rendering process; controlling the DMA controller to move the external data to an MxN storage area specified by the storage unit in batches so as to be processed by a graphic hardware acceleration engine; and the LCD controller is controlled to send the processed data to a display end according to a time sequence requirement of a display interface, so that final image display of the HMI display device is realized. The device is low in cost and excellent in performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital image signal processing, and in particular relates to an image processing display device, method and computer storage medium. Background Art

[0002] With the development of the automotive market, in-vehicle LCD instrument clusters are becoming increasingly popular and widely used in various vehicles. Dashboard information, such as vehicle speed, RPM, fuel level, entertainment, and warnings, is becoming increasingly diverse as people's needs grow. In the instrument clusters of small vehicles like two-wheeled motorcycles, and other HMI (Human Machine Interface) displays, the demand for UI displays is also growing. While high-performance graphical interfaces are required, the cost of display chips is also increasing to meet this market demand.

[0003] HMI displays offer a wide variety of interfaces, including RS232, RS485, and RJ45 network cables. Current HMI displays typically utilize a general-purpose GPU to modify the display frame buffer to achieve HMI graphics rendering. However, built-in general-purpose GPUs typically utilize larger SoC chips, requiring DDR and other memory for frame buffering, resulting in high costs and hindering market adoption. Furthermore, some simple HMI graphics displays utilize the CPU to modify display content directly, significantly impacting refresh rates due to the CPU's main frequency, limiting performance.

[0004] How to provide a low-cost, high-performance image processing and display device that meets HMI display requirements is a problem that needs to be solved urgently. Summary of the Invention

[0005] In view of this, the present invention provides an image processing and display device, method and computer storage medium to address the deficiencies of the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] An embodiment of the present invention provides an image processing and display device, comprising: a coordination control processor, a graphics hardware acceleration engine, an LCD controller, an SPI controller, a DMA controller, and a storage unit; each module component realizes interconnection control and data access in pairs through a bus interconnection matrix;

[0008] The SPI controller is used to perform high-speed, synchronous serial communication with an external directly connected memory to implement an operation of reading image data from the memory;

[0009] The DMA controller is used to directly move the image data read from the external memory by the SPI controller to the storage unit;

[0010] The graphics hardware acceleration engine is used to decode and render the source image data stored in the storage unit to obtain target image data;

[0011] The storage unit is used to store the fragmented source image data moved from the external memory and the destination image data after decoding and rendering processing by the graphics hardware acceleration engine;

[0012] The coordination control processor is used to coordinate the coordinated operation of various modules, deconstruct a single frame image into several MxN pixel regions through a pipeline segmentation mechanism, and call the graphics hardware acceleration engine in multiple rounds to perform rendering operations for each region based on a successive scheduling strategy;

[0013] The LCD controller is used to send the target image data stored in the storage unit to a display terminal connected to the LCD controller for display according to the timing requirements of the display interface.

[0014] Furthermore, the graphics hardware acceleration engine includes: a graphics acceleration engine master device, a graphics acceleration engine slave device, a graphics acceleration engine register, a first FIFO buffer, a second FIFO buffer, a third FIFO buffer and a graphics acceleration engine core;

[0015] The first FIFO buffer, the second FIFO buffer, and the third FIFO buffer are used to cache the source image, the reference image, and the destination image, respectively;

[0016] The graphics acceleration engine master device is interconnected with the bus interconnection matrix. The graphics acceleration engine master device reads source image data stored in the storage unit through the bus interconnection matrix and caches the data in a first FIFO buffer. Simultaneously, the graphics acceleration engine master device reads a previous frame of target image data stored in the storage unit and caches the data in a second FIFO buffer. The graphics acceleration engine master device reads the target image data cached in a third FIFO buffer and writes the data to a target image data storage area in the storage unit through the bus interconnection matrix.

[0017] The graphics acceleration engine slave device is interconnected with the bus interconnection matrix, and the coordination control processor accesses the graphics acceleration engine register through the graphics acceleration engine slave device via the bus interconnection matrix, and realizes software and hardware synchronization of graphics acceleration processing by configuring the graphics acceleration engine register parameters;

[0018] The graphics acceleration engine register is used to control and synchronize the working process of the graphics hardware acceleration engine to achieve image rendering in an MxN pixel area in multiple times. When the source image is moved from the external memory to the storage unit, the MxN pixel area has been allocated, and the MxN pixel area has not yet been rendered, the coordination control processor will configure the register related to the graphics acceleration engine kernel rendering entry in the graphics acceleration engine register through the graphics acceleration engine slave device. When the graphics acceleration engine kernel completes a rendering task, the status of the rendering exit is saved in the status register in the graphics acceleration engine register so that the coordination control processor can restore the rendering scene during the next rendering and continue decoding and rendering the image in the unrendered MxN pixel area.

[0019] The graphics acceleration engine core is used to perform rendering calculations on the source image data read from the first FIFO buffer and the reference image data read from the second FIFO buffer according to the mixing weights of the pixels and cache the calculated and processed data to be displayed in the third FIFO buffer.

[0020] Furthermore, the LCD controller includes: an LCD master device, an LCD slave device, an LCD buffer, a display output control processing unit, an LCD register, and an anti-tearing synchronization unit;

[0021] The LCD master device is interconnected with the bus interconnection matrix, and the LCD master device reads the MxN pixel data to be displayed stored in the target image data storage area of ​​the storage unit through the bus interconnection matrix, and buffers the data into the LCD buffer;

[0022] The LCD slave device is used to provide a slave device interface for coordinating the control processor's access to the LCD registers, thereby achieving software and hardware synchronization of the LCD controller driving the display;

[0023] The coordination control processor accesses the LCD register through the bus interconnection matrix and the LCD slave device, and realizes the software and hardware synchronization of the LCD controller driving the display by configuring the LCD register parameters;

[0024] The LCD buffer is used to buffer data read by the LCD master device for use by the display output control processing unit;

[0025] The anti-tearing synchronization unit is used to perform anti-tearing synchronization on the timing generated by the LCD controller using the TE signal fed back from the screen end;

[0026] The display output control processing unit is used to receive synchronization signals from the LCD register and the anti-tearing synchronization unit, read the display data in the LCD buffer, and send the display data to the display screen for display according to the standard interface timing.

[0027] Furthermore, the anti-tearing synchronization unit receives timing information set by the LCD register, including the total number of rows and dots of the display screen, as well as the valid rows, valid dots, and anti-tearing distance. When the anti-tearing timing counter is enabled, it waits for the TE signal. When the rising edge of the TE signal is detected, the row and dot counter starts working from the initial zero state. When the anti-tearing timing counter counts to the anti-tearing distance, the anti-tearing synchronization unit sends a control signal to the display output control processing unit to send the MxN display data to the display screen end.

[0028] Furthermore, the anti-tearing distance is set to a time length of an integer number of rows, and the interval between two rising edges of the TE signal is equal to the frame interval between two frames of images.

[0029] Furthermore, the anti-tearing synchronization unit simultaneously outputs the status of the current refresh line of the display screen. By judging the status of the refresh line, the display screen scan line can be determined, thereby determining the target area of ​​the image to which the LCD controller can actually transmit the update.

[0030] Furthermore, the bus interconnection matrix is ​​an AHB bus matrix.

[0031] An embodiment of the present invention provides an image processing and display device and an image processing and display method, which are applied to the above-mentioned device, and the method includes:

[0032] After receiving the image rendering instruction, the coordination control processor detects the TE signal fed back by the display end;

[0033] When the rising edge of the TE signal is detected, the LCD controller's timing monitoring task is started; at the same time, the MxN image decoding, rendering, and display processing task is started, and each frame of the image is divided into multiple MxN rectangular image blocks;

[0034] When the coordination control processor detects that the LCD controller meets the timing requirements, it calls the LCD controller to read the data from the MxN display memory and send it to the display screen;

[0035] Start the decoding and display processing task of the next MxN image in the frame until the decoding and rendering of the entire frame image are completed.

[0036] Furthermore, the MxN image decoding, rendering and display processing task specifically includes: decomposing the MxN image processing into K processing steps, configuring a DMA controller to move image data from an external memory to a storage unit; invoking a graphics hardware acceleration engine to read image source data cached in the storage unit for rendering processing, and writing the decoded and rendered image data into a destination image data storage area in the storage unit; determining whether decoding and rendering of the MxN rectangular image are complete; if decoding and rendering are not complete, determining whether to further configure the DMA controller to move image data from the external memory to a fragment buffer area of ​​the storage unit storing the source image based on the remaining size of the source image returned by the hardware acceleration engine, and continuing to execute the image decoding and rendering task until decoding and rendering of the MxN rectangular image are complete; and when detecting that the LCD controller timing meets timing requirements, invoking the LCD controller to read, process, and transmit the MxN rendered data to the display screen.

[0037] An embodiment of the present invention further provides a computer storage medium storing a computer program, wherein the computer program implements the above-mentioned image processing and display method when executed by a processor.

[0038] This invention utilizes segmented fragmentation and pipelined processing to achieve compressed image decoding and hardware-accelerated image rendering with minimal storage and bandwidth. It also provides an anti-tearing synchronization mechanism, preventing tearing when refreshing the display screen in MxN regions. This results in highly efficient graphics acceleration and high performance. This image processing device offers low cost, excellent performance, and significant economic value. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 A schematic structural diagram of an image processing and display device provided by an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the structure of the graphics hardware acceleration engine;

[0042] Figure 3 It is a structural diagram of the LCD controller;

[0043] Figure 4 The present invention provides a flowchart of an image processing and display method. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention. Figure 1 The figure shows the structure of an image processing and display device provided by an embodiment of the present invention. The device includes a coordinated control processor, an SPI controller, a DMA controller, a storage unit, a graphics hardware acceleration engine, and an LCD controller. Each module implements interconnection control and data access via a bus interconnection matrix. The AHB bus is preferably used for the bus interconnection matrix. The AHB bus is suitable for scenarios requiring efficient data transmission and low-latency communication, such as connecting processors and memories in embedded systems and controlling peripheral devices. The AHB bus is particularly suitable for applications such as automotive instrument clusters.

[0046] The SPI controller is used to perform high-speed, synchronous serial communication with an external directly connected memory to read image data from the memory. The DMA controller is used to directly transfer image data read from the external memory by the SPI controller to the storage unit. The DMA controller enables high-speed data transfer between the storage unit and the external memory without the intervention of the coordination control processor. The graphics hardware acceleration engine is used to decode and render the source image data stored in the storage unit to obtain target image data. The storage unit is used to store the fragmented source image data transferred from the external memory and the target image data decoded and rendered by the graphics hardware acceleration engine. The storage unit can be divided into at least two areas: one for storing the source image data transferred from the external memory, and the other for storing the target image data to be displayed after processing by the graphics hardware acceleration engine. The LCD controller is used to transmit the target image data stored in the storage unit to a display terminal connected to the LCD controller for display according to the display interface timing requirements. The coordination control processor is used to coordinate the coordinated operation of various modules, deconstructing a single frame image into several MxN pixel regions through a pipeline segmentation mechanism, and calling the graphics hardware acceleration engine in multiple rounds to perform rendering operations on each region based on a successive scheduling strategy. Each frame of image is divided into several MxN pixel areas, and the MxN pixel areas are decomposed into K processing times. Data is moved from the external memory in batches and the image rendering processing is performed using the graphics hardware acceleration engine.

[0047] The UI image source data of this device is stored in the external memory. Under the control of the coordinated control processor, the DMA controller fragments and segments the data of the external memory, such as the SPI flash, and moves it to the storage unit for processing by the graphics hardware acceleration engine. Finally, the LCD controller is called to send the processed data to the display end according to the display interface timing requirements, usually the CPU screen or serial port screen, to realize the final image display of the HMI.

[0048] The structure and working mechanism of the main modules are further described below to provide a better understanding of the present invention.

[0049] Graphics hardware acceleration engine, this module provides graphics hardware decoding acceleration processing function, its structure is as follows Figure 2 As shown, the graphics acceleration engine comprises a graphics acceleration engine master device, a graphics acceleration engine slave device, a graphics acceleration engine register, a first FIFO buffer (source image buffer), a second FIFO buffer (reference image buffer), a third FIFO buffer (destination image buffer), and a graphics acceleration engine core. The first FIFO buffer, the second FIFO buffer, and the third FIFO buffer are used to buffer the source image, the reference image, and the destination image, respectively.

[0050] The graphics acceleration engine master device is interconnected with a bus interconnection matrix. The graphics acceleration engine master device reads source image data stored in the storage unit in batches via the bus interconnection matrix, such as QOI compressed image data, and caches the source image data in a source image buffer. Simultaneously, it reads the previous frame of target image data stored in the storage unit and caches the target image data as a source reference frame in a reference image buffer. After processing by the graphics acceleration engine core, the graphics acceleration engine master device caches the target image data obtained after hardware decoding and rendering in a target image buffer, reads the data from the target image buffer, and writes it to the target image data storage area in the storage unit via the bus interconnection matrix.

[0051] The graphics acceleration engine slave device is interconnected with the bus interconnection matrix, providing a slave device interface for coordinating and controlling the processor's access to the graphics acceleration engine registers. The coordination control processor accesses the graphics acceleration engine registers through the graphics acceleration engine slave device via the bus interconnection matrix, and realizes software and hardware synchronization of graphics acceleration processing by configuring the graphics acceleration engine register parameters.

[0052] The graphics acceleration engine register is connected to the graphics acceleration engine slave device. The graphics acceleration engine register is used to control and synchronize the working process of the graphics hardware acceleration engine to achieve image rendering in multiple times in the MxN pixel area. When the qoi source image is moved from the external memory to the storage unit, and the new MxN pixel area has been allocated and the MxN pixel area has not yet been rendered, the coordination control processor will configure the register of the graphics acceleration engine kernel rendering entrance in the graphics acceleration engine register through the graphics acceleration engine slave device. When the graphics acceleration engine kernel completes a rendering task, it saves the status of the rendering exit in the status register in the graphics acceleration engine register, so that the coordination control processor can restore the rendering scene during the next rendering and continue to perform qoi decoding and rendering of the graphics in the unrendered area.

[0053] The main rendering entry and exit registers include: the remaining size of the source image asset_size, the source image pause address asset_addr, the output point coordinates x and y, the output point address pixel_addr and the first address of the row where the output point is located pixel_fifo, the rendering end status run_code, etc.

[0054] The graphics acceleration engine core reads the source image (qoi) from the source image buffer and simultaneously reads reference frame data from the reference image buffer. After performing rendering calculations based on pixel blending weights, the display data is sent to the destination image buffer and ultimately, via the graphics acceleration engine master device, to an MxN pixel destination data area in the storage unit via a bus interconnection matrix. Traditional image hardware acceleration engines, implemented using general-purpose GPUs, typically initiate tasks that render complete rectangles within a frame. Furthermore, general-purpose GPUs do not support compression decoding of images such as qoi. Image decoding is performed by a specialized decoder that first writes the image back to the frame buffer via the bus, then calls the GPU to read the complete frame data via the bus for hardware-accelerated rendering. Image decoding and rendering are separate processes, requiring more storage resources and bus bandwidth. The graphics hardware acceleration engine implemented in the present invention decodes compressed images using hardware while simultaneously rendering an MxN area, which can be smaller than a single frame. Rendering within the MxN area also supports pausing at any point in the output coordinates x and y. This allows for image decoding acceleration at a minimal cost, without requiring a full frame buffer, saving bus bandwidth. When a rendering task is completed, the status value run_code is returned. If run_code is 0, it means that the rendering of the MxN area is completely completed. If run_code is 1, it means that the rendering of the MxN area is not completed. When the task is started next time, it is necessary to read the status values, such as asset_size, asset_addr, x, y, pixel_addr, pixel_fifo, etc., to continue decoding and rendering the remaining images in the MxN area. When asset_size is 0, it means that the source data has been processed. If x and y have not reached the end of the MxN area at this time, new source data needs to be moved from the external memory to the storage unit to continue rendering the MxN area. The next starting value is configured according to the current asset_size, asset_addr, x, y, pixel_addr, pixel_fifo. On the contrary, if run_code is 0, x and y are already the last point, and the next rendering will start a new MxN rectangular area. At the same time, the next rendering will also read the values ​​of the status register asset_size, asset_addr, x, y, pixel_addr, and pixel_fifo to configure the next starting value. Through the above processing, the present invention aims to solve the problem of pipelined qoi decoding and image accelerated rendering and display in a segmented, fragmented, flexible and efficient manner under small storage conditions, thereby providing a highly competitive chip solution in the display field such as in-vehicle instruments to meet market demand.

[0055] The above describes the operation of the graphics hardware acceleration engine. When the MxN area has been rendered, the processor will schedule the LCD controller to read the rendered data at the appropriate time as needed and send it to the display end according to the timing of the CPU screen or serial port screen.

[0056] The structure of the LCD controller is as follows Figure 3 As shown, it includes: an LCD master device, an LCD slave device, an LCD register, an LCD buffer, a display output control processing unit and an anti-tearing synchronization unit.

[0057] The LCD master device is interconnected with the bus interconnection matrix and provides an interface for the LCD controller to read display data. The LCD master device reads the MxN display data stored in the storage unit through the bus interconnection matrix and caches it in the LCD buffer.

[0058] The LCD slave device is used to provide a slave device interface for coordinating the control processor's access to the LCD registers, thereby achieving software and hardware synchronization of the LCD controller driving the display.

[0059] The coordination control processor accesses LCD registers via the bus interconnect matrix and the LCD slave devices. By configuring the LCD register parameters, the coordination control processor synchronizes the hardware and software driving the LCD controller. For example, it configures the MxN display data area information, including address and size, and the corresponding update destination area on the display. After the LCD controller sends MxN data, it returns the status to the LCD status register, which the coordination control processor uses to coordinate the display of the next MxN display area.

[0060] The LCD buffer is used to cache data read from the LCD master device for use by the display output control processing unit.

[0061] Anti-tearing synchronization unit, the display screen end of the present invention is generally a CPU screen or an SPI serial port screen. In the drive chip of this type of screen, the most common problem encountered is the tearing phenomenon. Usually, the output timing of the main control end is generated according to its own display clock and a certain display standard (such as the VESA timing standard configuration). The present invention uses the TE signal (Tearing Effect Signal) fed back by the CPU screen or the SPI serial port screen end to perform anti-tearing synchronization on the timing generated by the LCD controller. The TE signal is a synchronization signal generated by the LCD controller, which is mainly used to solve the tearing effect (Tearing Effect) of the display screen. The TE signal achieves anti-tearing synchronization by coordinating the timing of data update and screen refresh. The TE signal is a hardware-level synchronization mechanism that fundamentally solves the tearing problem by coordinating the frame update of the image source and the screen refresh timing.

[0062] Specifically, the anti-tearing synchronization unit receives timing information set by the register module, including the total number of rows and dots of the display screen, as well as the valid rows, valid dots, and anti-tearing distance. When the anti-tearing timing counter is enabled, it waits for the TE signal. When the rising edge of the TE signal is detected, the row and dot counter starts working from the initial zero state. When the counter reaches the anti-tearing distance, the anti-tearing synchronization unit sends a control signal to the display output control processing unit to send MxN display data to the display end. Here, the anti-tearing distance can be set to the time length of an integer row. By setting this value, it can be ensured that the data written to the display end will not cause write and read conflicts with the display end driver circuit, thereby avoiding the occurrence of tearing. Between the two rising edges of the TE signal (the interval between the two rising edges is the frame interval time of two frames of images), the anti-tearing synchronization unit simultaneously outputs the state linepos representing the current refresh line of the screen. This allows the display screen to be accurately predicted when different MxN areas within a frame of image are transmitted, even when there is only a frame interval TE signal and no line interval TE signal. At the same time, tearing can be accurately avoided when the next MxN area is transmitted. In implementation, Linepos sets an offset register, lineofst, based on the row counter in the timing counter. Lineofst takes an integer value, and linepos is equal to the row counter plus lineofst. When starting an MxN region task, the linepos check determines the display scan line and, therefore, the actual destination area of ​​the image that the LCD controller can transmit to, thus avoiding tearing. For example, when the scan line is 8, the LCD controller can transmit eight rows of data, from rows 0 to 7.

[0063] The LCD controller's anti-tearing synchronization mechanism enables perfect display of instrument and other images, eliminating image distortion and tearing on low-cost CPU or serial port screen instrument solutions.

[0064] Display output control processing unit, this unit receives the synchronization signal of the register and anti-tearing synchronization, reads the display data in the LCD buffer, and sends the display data to the display screen for display according to the standard CPU screen or serial port screen interface timing (such as 8080 interface or spi, qspi interface, etc.), thereby completing the image decoding and rendering display functions of the present invention.

[0065] The present invention utilizes segmented fragmentation and pipeline processing to achieve compressed image decoding and hardware-accelerated image rendering with less storage and bandwidth. It also provides an anti-tearing synchronization mechanism, preventing tearing when refreshing the display screen in MxN regions. This results in high graphics acceleration efficiency and performance, and is of great value.

[0066] In the present invention, the DMA controller is a master device on the bus and is a universal DMA device. The SPI controller is a slave device on the bus and provides access to external storage and is also a universal device, so it will not be described in detail.

[0067] The device and method implemented by the present invention will be described in detail below with reference to the accompanying drawings.

[0068] The coordination control processor is responsible for the coordination control of each module in the entire HMI image rendering process and performs fragmented segmentation processing on the image. The hardware can be implemented with a small CPU or other processor. The workflow of the coordination control processor for image control processing is as follows: Figure 4 As shown, the following steps are included:

[0069] S1, the system enters the power-on reset state, each module is in the reset idle state after power-on, and the next step enters step S2;

[0070] S2, initialize the DMA controller, SPI controller, graphics hardware acceleration engine, LCD controller and other modules, and then proceed to step S3;

[0071] S3, waiting for graphics rendering instructions, after receiving the graphics rendering instructions, the next step goes to step S4.

[0072] S4, wait for TE signal. When the rising edge of TE signal is detected, start the timing monitoring task of LCD controller and enter S5 next.

[0073] S5. Start the MxN image decoding, rendering, and display processing task. MxN represents the image size, which can represent a rectangular image area that is smaller than or equal to one frame of image. M represents the width of the rectangular image, and N represents the height of the rectangular image. M and N are positive integers. The next step is S6.

[0074] S6: Decompose the MxN image decoding and rendering into K processing steps, and configure the DMA controller to move part of the image data from the external memory to the fragment buffer area of ​​the storage unit. Next, proceed to step S7.

[0075] S7: Call the graphics hardware acceleration engine, read the image source data cached in the fragment buffer area, decode the source data content, and write the decoded and rendered data into the MxN display storage area of ​​the storage unit. Next, proceed to step S8.

[0076] S8. Determine whether the MxN rectangle has been decoded and rendered. To achieve higher performance and lower costs, the present invention utilizes fragmented storage space for only a portion of a single image frame. To meet refresh rate requirements, this fragmented storage space is utilized, employing a segmented pipeline processing approach to divide a single image frame into multiple MxN regions. Within each MxN sub-region, the graphics hardware acceleration engine can be invoked multiple times, ultimately achieving image rendering using a fragmented, highly efficient pipeline processing approach. Therefore, in step S8, a determination is made as to whether the MxN rectangle has been decoded and rendered. If decoding and rendering are not complete, the process proceeds to step S9. If they are, the process proceeds to step S10.

[0077] S9: Based on the status of the remaining size of the MxN area returned by the graphics hardware acceleration engine, determine whether to continue configuring DMA to move the compressed image data from the external memory to the fragment buffer area. The next step is to proceed to step S7 to continue the current image decoding and rendering process.

[0078] S10: When the LCD controller detects that the timing monitoring task is enabled, the LCD controller is called to read data from the MxN display memory, process it, and send it to the display screen. This step ensures that the LCD controller's timing monitoring task is enabled, preventing tearing (i.e., image distortion caused by conflicts between reading and writing display data) when the display data is output to the screen. The next step proceeds to step S11.

[0079] S11: Whether decoding and rendering of a frame of image is completed. This step determines whether decoding and rendering of a frame of image is completed. If not, the process proceeds to step S12; if completed, the process proceeds to step S13.

[0080] S12: Start the next MxN image decoding and display processing task within a frame, and then proceed to step S6.

[0081] S13, waiting for the next frame processing instruction, entering the next frame data processing and transmission process when the next frame is started, and entering step S4 to repeat the subsequent frame image decoding and rendering process until the display is completed.

[0082] The description of the workflow of the coordinated control processor provides a detailed understanding of the implementation steps of the present invention. The present invention also provides a computer storage medium, which is a computer-readable storage medium and stores a computer program. The computer program can be executed by the coordinated control processor to complete the steps of the aforementioned image processing and display method. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface mount storage device, an optical disk, or a compact disc read-only memory (CD-ROM).

[0083] In the several embodiments provided by the present invention, it should be understood that the disclosed methods and intelligent devices can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0084] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An image processing and display device, characterized in that: include: Coordinate control processor, graphics hardware acceleration engine, LCD controller, SPI controller, DMA controller, storage unit; each module component realizes interconnection control and data access through bus interconnection matrix; The SPI controller is used to perform high-speed, synchronous serial communication with an external directly connected memory to implement an operation of reading image data from the memory; The DMA controller is used to directly move the image data read from the external memory by the SPI controller to the storage unit; The graphics hardware acceleration engine is used to decode and render the source image data stored in the storage unit to obtain target image data; The storage unit is used to store the fragmented source image data moved from the external memory and the destination image data after decoding and rendering processing by the graphics hardware acceleration engine; The coordination control processor is used to coordinate the coordinated operation of various modules, deconstruct a single frame image into several MxN pixel regions through a pipeline segmentation mechanism, and call the graphics hardware acceleration engine in multiple rounds to perform rendering operations for each region based on a successive scheduling strategy; The LCD controller is used to send the target image data stored in the storage unit to a display terminal connected to the LCD controller for display according to the timing requirements of the display interface.

2. The device according to claim 1, characterized in that The graphics hardware acceleration engine includes: a graphics acceleration engine master device, a graphics acceleration engine slave device, a graphics acceleration engine register, a first FIFO buffer, a second FIFO buffer, a third FIFO buffer and a graphics acceleration engine core; The first FIFO buffer, the second FIFO buffer, and the third FIFO buffer are used to cache the source image, the reference image, and the destination image, respectively; The graphics acceleration engine master device is interconnected with the bus interconnection matrix. The graphics acceleration engine master device reads source image data stored in the storage unit through the bus interconnection matrix and caches the data in a first FIFO buffer. Simultaneously, the graphics acceleration engine master device reads a previous frame of target image data stored in the storage unit and caches the data in a second FIFO buffer. The graphics acceleration engine master device reads the target image data cached in a third FIFO buffer and writes the data to a target image data storage area in the storage unit through the bus interconnection matrix. The graphics acceleration engine slave device is interconnected with the bus interconnection matrix, and the coordination control processor accesses the graphics acceleration engine register through the graphics acceleration engine slave device via the bus interconnection matrix, and realizes software and hardware synchronization of graphics acceleration processing by configuring the graphics acceleration engine register parameters; The graphics acceleration engine register is used to control and synchronize the working process of the graphics hardware acceleration engine to achieve image rendering in an MxN pixel area in multiple times. When the source image is moved from the external memory to the storage unit, the MxN pixel area has been allocated, and the MxN pixel area has not yet been rendered, the coordination control processor will configure the register related to the graphics acceleration engine kernel rendering entry in the graphics acceleration engine register through the graphics acceleration engine slave device. When the graphics acceleration engine kernel completes a rendering task, the status of the rendering exit is saved in the status register in the graphics acceleration engine register so that the coordination control processor can restore the rendering scene during the next rendering and continue decoding and rendering the image in the unrendered MxN pixel area. The graphics acceleration engine core is used to perform rendering calculations on the source image data read from the first FIFO buffer and the reference image data read from the second FIFO buffer according to the mixing weights of the pixels and cache the calculated and processed data to be displayed in the third FIFO buffer.

3. The device according to claim 1, characterized in that The LCD controller includes: an LCD master device, an LCD slave device, an LCD buffer, a display output control processing unit, an LCD register, and an anti-tearing synchronization unit; The LCD master device is interconnected with the bus interconnection matrix, and the LCD master device reads the MxN pixel data to be displayed stored in the target image data storage area of ​​the storage unit through the bus interconnection matrix, and buffers the data into the LCD buffer; The LCD slave device is used to provide a slave device interface for coordinating the control processor's access to the LCD registers, thereby achieving software and hardware synchronization of the LCD controller driving the display; The coordination control processor accesses the LCD register through the bus interconnection matrix and the LCD slave device, and realizes the software and hardware synchronization of the LCD controller driving the display by configuring the LCD register parameters; The LCD buffer is used to buffer data read by the LCD master device for use by the display output control processing unit; The anti-tearing synchronization unit is used to perform anti-tearing synchronization on the timing generated by the LCD controller using the TE signal fed back from the screen end; The display output control processing unit is used to receive synchronization signals from the LCD register and the anti-tearing synchronization unit, read the display data in the LCD buffer, and send the display data to the display screen for display according to the standard interface timing.

4. The device according to claim 3, characterized in that The anti-tearing synchronization unit receives the timing information set by the LCD register, including the total number of rows and dots of the display screen, as well as the effective rows, effective dots, and anti-tearing distance. When the anti-tearing timing counter is enabled, it waits for the TE signal. When the rising edge of the TE signal is detected, the row and dot counter starts working from the initial zero state. When the anti-tearing timing counter counts to the anti-tearing distance, the anti-tearing synchronization unit sends a control signal to the display output control processing unit to send the MxN display data to the display screen end.

5. The device according to claim 4, characterized in that The anti-tearing distance is set to the time length of an integer line, and the interval between two rising edges of the TE signal is equal to the frame interval between two frames of images.

6. The device according to claim 4, characterized in that The anti-tearing synchronization unit simultaneously outputs the status of the current refresh line of the display screen. By judging the status of the refresh line, the display screen scan line can be determined, thereby determining the target area of ​​the image to which the LCD controller can actually transmit the update.

7. The device according to claim 1, characterized in that The bus interconnection matrix is ​​an AHB bus matrix.

8. An image processing and display method, applied to the device according to any one of claims 1 to 7, characterized in that: The method comprises: After the system is powered on, the DMA controller, SPI controller, graphics hardware acceleration engine and LCD controller are initialized; After receiving the image rendering instruction, the coordination control processor detects the TE signal fed back by the display end; When the rising edge of the TE signal is detected, the LCD controller's timing monitoring task is started; at the same time, the MxN image decoding, rendering, and display processing task is started, and each frame of the image is divided into multiple MxN rectangular image blocks; When the coordination control processor detects that the LCD controller meets the timing requirements, it calls the LCD controller to read the data from the MxN display memory and send it to the display screen; Start the decoding and display processing task of the next MxN image in the frame until the decoding and rendering of the entire frame image are completed.

9. The method according to claim 8, characterized in that The MxN image decoding, rendering and display processing task specifically includes: decomposing the MxN image processing into K processing steps, configuring a DMA controller to move image data from an external memory to the storage unit; invoking a graphics hardware acceleration engine to read image source data cached in the storage unit for rendering processing, and writing the decoded and rendered image data into a destination image data storage area in the storage unit; determining whether decoding and rendering of the MxN rectangular image are complete; if decoding and rendering are not complete, determining whether to further configure the DMA controller to move image data from the external memory to a fragment buffer area of ​​the storage unit storing the source image based on the remaining size of the source image returned by the hardware acceleration engine, and continuing to execute the image decoding and rendering task until decoding and rendering of the MxN rectangular image are complete; and, when detecting that the LCD controller timing meets timing requirements, invoking the LCD controller to read, process, and transmit the MxN rendered data to the display screen.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing and display method according to any one of claims 8 to 9 is implemented.

Images