A highly integrated graphics generation and processing device in a head-up display system

Abstract

The application provides a highly integrated graphics generation and processing device in a head-up display system. The device comprises a drawing instruction cache module, a graphics drawing module, a Cache cache module, a display output module, a video synchronization module and an image superposition module. The application is implemented based on FPGA, has not only a graphics generation function, but also integrated picture distortion correction function and image superposition function. The application has the advantages of clear structure, low complexity, high integration, low delay, low power consumption and the like, and has wide application value in miniaturized head-up display devices.

Description

Technical Field

[0001] This invention belongs to the field of graphics generation technology, and specifically relates to a highly integrated graphics generation and processing device for a head-up display system. Background Technology

[0002] Airborne display systems generate symbolic images based on received flight parameters and display them on head-up displays (HUDs). Currently, commercial GPUs are used to generate these symbols; however, commercial GPUs have high power consumption and large size, and require additional circuitry for distortion correction and image overlay functions. Therefore, there is an urgent need for a low-power, highly integrated graphics generation and processing device for HUD systems. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, this invention provides a highly integrated graphics generation and processing device for a head-up display system. It includes a drawing instruction cache module, a graphics drawing module, a cache module, a display output module, a video synchronization module, and an image overlay module. It not only has graphics generation capabilities but also integrates image distortion correction and image overlay functions. This invention features a simple structure, low complexity, high integration, low latency, and low power consumption.

[0004] A highly integrated graphics generation and processing device for a head-up display system, characterized in that it includes: a drawing instruction cache module, a graphics drawing module, a cache module, a display output module, a video synchronization module, and an image overlay module, all of which are implemented by FPGA; wherein, the image drawing module and the video synchronization module respectively integrate distortion correction functions;

[0005] The drawing instruction cache module receives drawing instructions sent by the graphics rendering processor via an external bus. It can cache up to two frames of drawing instructions. After the graphics rendering processor sends one frame of drawing instructions, it sends a switch cache flag and stores it in the drawing instruction cache module.

[0006] The graphics drawing module reads drawing instructions from the drawing instruction cache module, completes the drawing of graphics and performs distortion correction according to the drawing instructions, and then reads the color value in the current external graphics memory address through the Cache cache module, mixes it with the color value of the corresponding pixel coordinate in the distortion-corrected drawing data according to the transparency coefficient, and then writes it into the external graphics memory.

[0007] The display output module reads drawing data from external graphics memory and outputs the display image to the image overlay module according to the VESA timing of the set resolution.

[0008] The video synchronization module performs distortion correction on the video data in the external EVS video clock domain and outputs it to the image overlay module, and synchronizes it to the clock domain of the display output module.

[0009] The image overlay module overlays the synchronized EVS video and the display screen output by the display output module before outputting the result.

[0010] Specifically, the external bus is a PCIE or AXI bus.

[0011] Specifically, the drawing instructions include setting commands and graphics commands. The setting commands mainly include parameter settings such as graphics resolution, window opening, and occlusion area. The graphics commands mainly include drawing instructions for primitive types such as points, lines, circles, triangles, arcs, characters, and Bézier curves, as well as primitive drawing parameters including color, line width, and transparency.

[0012] Specifically, the graphics rendering module reads the drawing instructions and the graphics rendering processor sends the drawing instructions asynchronously. When the graphics rendering processor sends the drawing instructions faster than the graphics rendering module reads the drawing instructions, there are two switching cache flags in the drawing instruction cache module at the same time. The drawing instruction cache module will no longer receive the new frame of drawing instructions until the graphics rendering module reads a switching cache flag, after which the drawing instruction cache module will accept the new frame of drawing instructions. When the graphics rendering processor sends the drawing instructions slower than the graphics rendering module reads the drawing instructions, the graphics rendering module stops drawing and waits for the graphics rendering processor to send new drawing instructions.

[0013] Specifically, the external graphics memory adopts a dual-buffering mechanism, divided into a drawing data buffer A and a drawing data buffer B. When the graphics drawing module writes all the drawing data of a frame into the drawing data buffer A, if the display output module is reading the drawing data buffer B at the same time, the graphics drawing module will wait until the display output module finishes reading the drawing data buffer B before starting to write the next frame of drawing data into the data buffer B. Otherwise, the graphics drawing module will directly write the next frame of drawing data into the data buffer B. When the display output module finishes reading the drawing data buffer A, if the graphics drawing module is writing to the data buffer B at the same time, the display output module will continue to read the drawing data buffer A. Otherwise, the display output module will read the drawing data buffer B.

[0014] Specifically, the distortion correction function in the image drawing module and video synchronization module refers to performing distortion correction processing on the coordinates of the drawing data generated by the graphics drawing module and the video data in the external EVS video clock domain, and using the nearest distance coordinate point after processing as the final pixel coordinates; the distortion correction formula is the fitting equation formula B = f(A) obtained based on the distortion characteristics of the optical system, where A represents the input coordinate data, B represents the output coordinate data, and f is the transformation function.

[0015] Specifically, the image overlay module performs overlay processing according to the formula C = (A + B) - A × B / 255, where A is the EVS video pixel value, B is the pixel value of the output screen of the display output module, and C is the pixel value of the image after overlay processing.

[0016] Specifically, the output screen includes EVS video, the screen output by the display output module, and the screen output by the image overlay module.

[0017] The beneficial effects of the present invention are: (1) It has low complexity and is easy to implement in hardware; (2) It has high integration and can realize the graphics generation function, distortion correction function and image overlay function simultaneously based on FPGA; (3) It has low latency and adopts a pipelined data transmission and storage mechanism; (4) It has low power consumption and the FPGA device has low power consumption. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of a highly integrated graphics generation and processing device in a head-up display system according to the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments. The present invention includes, but is not limited to, the following embodiments.

[0020] like Figure 1 As shown, this invention provides a highly integrated graphics generation and processing device for a head-up display system, comprising: a drawing instruction cache module, a graphics drawing module, a cache module, a display output module, a video synchronization module, and an image overlay module. The image drawing module and the video synchronization module each integrate distortion correction functions. This invention's graphics generation and processing device not only has graphics generation capabilities but also integrates image distortion correction and image overlay functions.

[0021] The drawing instruction cache module receives drawing instructions sent by the graphics rendering processor via an external bus. It can cache a maximum of two frames of drawing instructions. After each frame of drawing instructions is sent, the graphics rendering processor sends a cache switch flag, which is then stored in the drawing instruction cache module. The bus between this invention and the graphics rendering processor is typically a high-speed bus such as PCIe or AXI. Drawing instructions generally include setting commands and drawing commands. Setting commands include parameter settings such as graphics resolution, window opening, and occlusion; drawing commands include drawing commands for primitive types such as points, lines, circles, triangles, arcs, characters, and Bézier curves, as well as primitive drawing parameters such as color, line width, and transparency.

[0022] The graphics rendering module reads drawing instructions and the graphics rendering processor sends drawing instructions asynchronously. When the graphics rendering processor sends drawing instructions faster than the graphics rendering module reads them, two switching buffer flags exist simultaneously in the drawing instruction buffer module. The drawing instruction buffer module will not receive new frame drawing instructions until the graphics rendering module reads one switching buffer flag, after which it will accept new frame drawing instructions. When the graphics rendering processor sends drawing instructions slower than the graphics rendering module reads them, the graphics rendering module stops drawing and waits for the graphics rendering processor to send new drawing instructions.

[0023] The graphics drawing module reads drawing instructions from the drawing instruction cache module, completes the setting of graphics parameters such as graphics resolution, window opening, and occlusion area, and then completes the drawing of basic graphic elements such as points, lines, circles, triangles, arcs, characters, and Bézier curves according to the graphics commands.

[0024] The display output of this invention needs to be displayed through a head-up display (HUD), which contains a series of optical systems. If the display image is not processed, the image perceived by the human eye will be severely distorted after passing through the optical system. Therefore, distortion correction processing is required based on the distortion characteristics of the optical system. The disadvantages of traditional FPGA-based distortion correction methods are: 1. They not only occupy internal Block RAM resources of the FPGA but also occupy FPGA pin resources to connect to external memory. 2. The distortion correction process has a millisecond-level delay. Therefore, this invention adopts a new distortion correction processing method in the graphics drawing module. A fitting equation formula B = f(A) is generated based on the distortion characteristics of the optical system, where A represents the input coordinate data, B represents the output coordinate data, and f is the transformation function. The drawing data generated by the graphics drawing module consists of the original pixel coordinates (xa, ya) and color value (color). After calculation using the distortion fitting equation formula, the distorted pixel coordinates (xb, yb) are obtained. Since the coordinates (xb, yb) are decimals, the nearest integer coordinates (xc, yc) are found using the least nearest neighbor method. Finally, the graphics rendering module stores the distorted pixel coordinates (xc, yc) and pixel value (color) into external graphics memory.

[0025] The graphics rendering module stores color values ​​into the corresponding external graphics memory address based on the pixel coordinates of the generated drawing data. Before storing the color value, it needs to read the color value from the current external graphics memory address. The two color values ​​are then mixed according to their transparency coefficients before being written to the external graphics memory. This avoids the phenomenon where a lower color value written later overwrites a higher color value written earlier at the same address, which could lead to display defects. To improve the efficiency of the graphics rendering module accessing the external graphics memory, a cache module is added between the two to improve the readback mechanism.

[0026] The transparency factor is included in the drawing instructions sent by the graphics rendering processor; generally, primitives of the same type will have the same transparency factor. Assume the color value generated by the graphics rendering module is C. S The transparency data of the currently drawn graphic in the graphics drawing module is A. S The current color value in external video memory is C. d The transparency in the external video memory is A. d The final color value for the transparency coefficient blend is:

[0027] C = (C S ·A S +C d ·A d (1-A S )) / A (1)

[0028] Among them, A=1-(1-A S )·(1-A d ).

[0029] The external graphics memory employs a dual-buffering mechanism, divided into drawing data buffers A and B. When the graphics rendering module writes all the drawing data for a frame into drawing data buffer A, if the display output module is currently reading from drawing data buffer B, it will wait until the display output module has finished reading from buffer B before starting to write the next frame of drawing data into buffer B. Otherwise, the graphics rendering module directly writes the next frame of drawing data into buffer B. When the display output module has finished reading from drawing data buffer A, if the graphics rendering module is currently writing to buffer B, it will continue reading from drawing data buffer A; otherwise, it will read from drawing data buffer B. This mechanism ensures the integrity of each frame of drawing data.

[0030] The display output module reads drawing data from external graphics memory and outputs the display image to the image overlay module according to the VESA timing of the set resolution.

[0031] The video synchronization module performs distortion correction on the video data in the external EVS video clock domain and synchronizes it to the clock domain of the display output module, facilitating subsequent overlay processing of the two images. The video synchronization module also integrates distortion correction functionality, implemented in the same way as the distortion correction function integrated in the graphics rendering module.

[0032] Once the EVS video and the display output module output images are synchronized, image overlay can be achieved. You can choose to output the EVS video, the image output by the display output module, or an overlay of both images.

[0033] Image overlay is achieved by superimposing pixel-by-pixel data between the EVS video and the output image from the display output module, using the formula C = (A + B) - A × B / 255. Where: A is the pixel value of the EVS video, B is the pixel value of the output image from the display output module, and C is the pixel value of the overlaid image.

Claims

1. A highly integrated graphics generation and processing device for a head-up display system, characterized in that... include: The drawing instruction cache module, graphics drawing module, cache module, display output module, video synchronization module, and image overlay module are all implemented by FPGA; among them, the image drawing module and video synchronization module integrate distortion correction functions. The drawing instruction cache module receives drawing instructions sent by the graphics rendering processor via an external bus. It can cache up to two frames of drawing instructions. After the graphics rendering processor sends one frame of drawing instructions, it sends a switch cache flag and stores it in the drawing instruction cache module. The graphics drawing module reads drawing instructions from the drawing instruction cache module, completes the graphics drawing and performs distortion correction according to the drawing instructions, and then reads the color value in the current external graphics memory address through the cache module. It mixes the color value of the corresponding pixel coordinates in the distortion-corrected drawing data with the color value of the corresponding pixel coordinates according to the transparency coefficient before writing it into the external graphics memory. The distortion correction function in the image drawing module and the video synchronization module refers to performing distortion correction processing on the coordinates of the drawing data generated by the graphics drawing module and the video data in the external EVS video clock domain, and using the nearest distance coordinate point after processing as the final pixel coordinates. The distortion correction formula is the fitting equation B = f(A) obtained based on the distortion characteristics of the optical system, where A represents the input coordinate data, B represents the output coordinate data, and f is the transformation function. The display output module reads drawing data from external graphics memory and outputs the display image to the image overlay module according to the VESA timing of the set resolution. The video synchronization module performs distortion correction on the video data in the external EVS video clock domain and outputs it to the image overlay module, and synchronizes it to the clock domain of the display output module. The image overlay module overlays the synchronized EVS video and the display screen output by the display output module before outputting the result. The overlay process of the image overlay module is calculated pixel by pixel according to C = (A + B) - A × B / 255, where A is the pixel value of the EVS video, B is the pixel value of the display screen output by the display output module, and C is the pixel value of the overlay image.

2. The highly integrated graphics generation and processing device in a head-up display system as described in claim 1, characterized in that: The external bus is either a PCIE or AXI bus.

3. The highly integrated graphics generation and processing device in a head-up display system as described in claim 1, characterized in that: The drawing commands include setting commands and graphics commands. The setting commands include parameter settings such as graphics resolution, window opening, and occlusion area. The graphics commands include drawing commands for primitive types such as points, lines, circles, triangles, arcs, characters, and Bézier curves, as well as primitive drawing parameters including color, line width, and transparency.

4. The highly integrated graphics generation and processing device in a head-up display system as described in claim 1, characterized in that: The graphics rendering module reads the drawing instructions and the graphics rendering processor sends the drawing instructions asynchronously. When the graphics rendering processor sends the drawing instructions faster than the graphics rendering module reads the drawing instructions, there are two switching cache flags in the drawing instruction cache module at the same time. The drawing instruction cache module will no longer receive the new frame of drawing instructions until the graphics rendering module reads a switching cache flag, after which the drawing instruction cache module will accept the new frame of drawing instructions. When the graphics rendering processor sends the drawing instructions slower than the graphics rendering module reads the drawing instructions, the graphics rendering module stops drawing and waits for the graphics rendering processor to send new drawing instructions.

5. The highly integrated graphics generation and processing device in a head-up display system as described in claim 1, characterized in that: The external graphics memory employs a dual-buffering mechanism, consisting of a graphics data buffer A and a graphics data buffer B. When the graphics rendering module writes all the graphics data of a frame into the graphics data buffer A, if the display output module is currently reading from the graphics data buffer B, it will wait until the display output module has finished reading from the graphics data buffer B before starting to write the next frame of graphics data into the data buffer B. Otherwise, the graphics rendering module will directly write the next frame of graphics data into the data buffer B. When the display output module has finished reading from the graphics data buffer A, if the graphics rendering module is currently writing to the data buffer B, the display output module will continue reading from the graphics data buffer A; otherwise, the display output module will read from the graphics data buffer B.

6. The highly integrated graphics generation and processing device in a head-up display system as described in claim 1, characterized in that: The output screen includes EVS video, the screen output by the display output module, and the screen output by the image overlay module.

Images