An integrated graphics generation and distortion processing apparatus in a head-up display system

Abstract

The application discloses a kind of integrated graphic generation and distortion processing device in head-up display system, including drawing instruction cache module, graphics drawing module, Cache cache module, display output module, video synchronization module and image superposition module;Drawing instruction cache module receives the drawing instruction sent by graphics drawing processor through external bus;Graphics drawing module reads the drawing instruction from drawing instruction cache module, completes primitive drawing;Cache cache module is added between graphics drawing module and graphics memory;Display output module completes the primitive drawing data from external graphics memory and reads out, video synchronization module synchronizes the video data of external EVS video clock domain into the clock domain of display output module, and display output module is output and EVS video is superimposed and processed;EVS video and display output module output picture synchronization, that is, image superposition output is realized.The application has lower complexity, is easy to hardware implementation, and high integration.

Description

Technical Field

[0001] This invention belongs to the field of airborne display technology, specifically relating to an integrated graphics generation and distortion processing device in a head-up display system. Background Technology

[0002] The airborne display system generates symbolic images based on received flight parameters and displays them on head-up and screen displays. Currently, commercial GPUs are used to generate these symbolic images, but these GPUs have high power consumption and large size, and require additional circuitry to perform distortion correction and image overlay functions.

[0003] Therefore, there is an urgent need for a low-power, highly integrated graphics generation and processing device for head-up display systems. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention provides an integrated graphics generation and distortion processing device for a head-up display system, including a drawing instruction cache module, a graphics drawing module, a cache module, a display output module, a video synchronization module, and an image overlay module. The drawing instruction cache module receives drawing instructions sent by the graphics drawing processor via an external bus. The graphics drawing module reads the drawing instructions from the drawing instruction cache module and completes the drawing of graphics primitives. A cache module is added between the graphics drawing module and the graphics memory. The display output module reads the graphics primitive drawing data from the external graphics memory. The video synchronization module synchronizes the video data from the external EVS video clock domain to the clock domain of the display output module. The display output module outputs the video data and the EVS video data for overlay processing. After the EVS video and the display output module output images are synchronized, image overlay output is achieved. This invention has low complexity, is easy to implement in hardware, and has high integration.

[0005] The technical solution adopted by this invention to solve its technical problem is as follows:

[0006] An integrated graphics generation and distortion processing device for a head-up display system includes a drawing instruction cache module, a graphics drawing module, a cache module, a display output module, a video synchronization module, and an image overlay module;

[0007] The drawing instruction cache module receives drawing instructions sent by the graphics rendering processor via an external bus;

[0008] The graphics drawing module reads drawing instructions from the drawing instruction cache module, completes the setting of graphics parameters such as graphics resolution, window opening, and occlusion area, and then completes the drawing of primitives according to the graphics commands; the graphics drawing module integrates distortion correction function;

[0009] The graphics drawing module stores the color value into the corresponding address of the external graphics memory based on the coordinates of the pixel points of the generated primitive drawing data. Before storing the color value, it reads the color value in the current external graphics memory address, and then writes the two color values ​​into the external graphics memory after weighted mixing. A cache module is added between the graphics drawing module and the graphics memory to improve the efficiency of the graphics drawing module in accessing the external graphics memory.

[0010] The display output module reads the primitive drawing data from the external graphics memory and outputs the display screen according to the VESA timing of the preset resolution.

[0011] The video synchronization module synchronizes the video data from the external EVS video clock domain to the clock domain of the display output module, which is used for superimposing the output of the display output module and the EVS video; the video synchronization module integrates distortion correction function.

[0012] Once the output images from the EVS video and display output modules are synchronized, image overlay output is achieved.

[0013] Preferably, the external bus is a PCIE or AXI bus.

[0014] Preferably, the graphic commands include drawing instructions for point, line, circle, triangle, arc, character, and Bézier curve primitive types, as well as drawing parameters for color and line width primitives.

[0015] Preferably, the interaction process between the graphics rendering processor, the drawing instruction cache module, and the graphics rendering module is as follows:

[0016] After the graphics rendering processor sends out a frame of drawing instructions, it sends a switch buffer flag and stores it in the drawing instruction buffer module. The drawing instruction buffer module can buffer a maximum of two frames of drawing instructions.

[0017] The graphics rendering processor sends drawing commands and the graphics rendering module reads drawing commands asynchronously. When the graphics rendering processor sends drawing commands quickly and the graphics rendering module reads drawing commands from the drawing command cache module slowly, there will be two switching cache flags in the drawing command cache module at the same time. At this time, the drawing command cache module will no longer receive the new frame of drawing commands until the graphics rendering module reads out one of the switching cache flags, and then the drawing command cache module will accept the new frame of drawing commands.

[0018] When the graphics rendering processor sends drawing instructions slowly, but the graphics rendering module reads drawing instructions quickly, the address of the drawing instructions read by the graphics rendering module will catch up with the address of the drawing instructions sent by the graphics rendering processor. At this time, the graphics rendering module stops drawing and waits for the graphics rendering processor to send new drawing instructions; this mechanism ensures the integrity of each frame of drawing instructions.

[0019] Preferably, the distortion correction function specifically includes:

[0020] Based on the distortion characteristics of the optical system, a fitting equation B = f(A) is generated. The drawing data generated by the graphics rendering module consists of the original pixel coordinates (xa, ya) and color value (color). After calculation using the distortion fitting equation, the distorted pixel coordinates (xb, yb) are obtained. Since the coordinates (xb, yb) are decimals, the nearest integer coordinates (xc, yc) are found using the least nearest neighbor method. Finally, the graphics rendering module stores the distorted pixel coordinates (xc, yc) and pixel value (color) in the external graphics memory.

[0021] Preferably, the external graphics memory employs a dual-buffering mechanism, divided into drawing data buffers A and B. When the graphics rendering module writes all the drawing data of a frame into drawing data buffer A, if the display output module is currently reading drawing data buffer B, the graphics rendering module will wait until the display output module finishes reading drawing data buffer B before starting to write the next frame of drawing data into data buffer B; otherwise, the graphics rendering module will directly write the next frame of drawing data into data buffer B. When the display output module finishes reading drawing data buffer A, if the graphics rendering module is currently writing into data buffer B, the display output module will continue reading drawing data buffer A; otherwise, the display output module will read drawing data buffer B. This mechanism ensures the integrity of each frame of drawing data.

[0022] Preferably, the image overlay output can select to output EVS video separately, output the screen of the display output module separately, or output an overlay of both images.

[0023] Preferably, the image overlay is achieved by superimposing the EVS video and the output screen of the display output module pixel by pixel, with the formula C = (D + E) - D × E / 255; where: D is the pixel value of the EVS video, E is the pixel value of the output screen of the display output module, and C is the pixel value of the overlaid image.

[0024] The beneficial effects of this invention are as follows:

[0025] 1. Low complexity, easy to implement in hardware.

[0026] 2. High integration: Based on FPGA, it implements graphics generation, distortion correction, and image overlay functions.

[0027] 3. Low latency, pipelined data transmission and storage mechanism.

[0028] 4. Low power consumption: FPGA devices have low power consumption. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the system architecture of the present invention. Detailed Implementation

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0031] The purpose of this invention is to construct a highly integrated graphics generation and distortion processing device for a head-up display system, which not only has graphics generation function, but also integrates image distortion correction function and image overlay function.

[0032] An integrated graphics generation and distortion processing device for a head-up display system includes a drawing instruction cache module, a graphics drawing module, a cache module, a display output module, a video synchronization module, and an image overlay module;

[0033] The drawing instruction cache module receives drawing instructions sent by the graphics rendering processor via an external bus; the external bus is a PCIE or AXI bus.

[0034] The graphics drawing module reads drawing instructions from the drawing instruction cache module, completes the setting of graphics parameters such as graphics resolution, window opening, and occlusion area, and then completes the drawing of graphic elements according to the graphics commands. The graphics drawing module integrates distortion correction function. The graphics commands include drawing instructions for point, line, circle, triangle, arc, character, and Bézier curve graphic elements, as well as color and line width graphic element drawing parameters.

[0035] After the graphics rendering processor sends out a frame of drawing instructions, it sends a switch buffer flag and stores it in the drawing instruction buffer module. The drawing instruction buffer module can buffer a maximum of two frames of drawing instructions.

[0036] The graphics rendering processor sends drawing commands and the graphics rendering module reads drawing commands asynchronously. When the graphics rendering processor sends drawing commands quickly and the graphics rendering module reads drawing commands from the drawing command cache module slowly, there will be two switching cache flags in the drawing command cache module at the same time. At this time, the drawing command cache module will no longer receive the new frame of drawing commands until the graphics rendering module reads out one of the switching cache flags, and then the drawing command cache module will accept the new frame of drawing commands.

[0037] When the graphics rendering processor sends drawing instructions slowly, but the graphics rendering module reads drawing instructions quickly, the address of the drawing instructions read by the graphics rendering module will catch up with the address of the drawing instructions sent by the graphics rendering processor. At this time, the graphics rendering module stops drawing and waits for the graphics rendering processor to send new drawing instructions; this mechanism ensures the integrity of each frame of drawing instructions.

[0038] Based on the distortion characteristics of the optical system, a fitting equation B = f(A) is generated. The graphics rendering module generates the original pixel coordinates (xa, ya) and color value color. After calculation using the distortion fitting equation, the distorted pixel coordinates (xb, yb) are obtained. Since the coordinates (xb, yb) are decimals, the nearest integer coordinates (xc, yc) are found using the least nearest neighbor method. Finally, the graphics rendering module stores the distorted pixel coordinates (xc, yc) and pixel value color into the external graphics memory.

[0039] The graphics drawing module stores the color value into the corresponding address of the external graphics memory based on the coordinates of the pixel points of the generated primitive drawing data. Before storing the color value, it reads the color value in the current external graphics memory address, and then writes the two color values ​​into the external graphics memory after weighted mixing. A cache module is added between the graphics drawing module and the graphics memory to improve the efficiency of the graphics drawing module in accessing the external graphics memory.

[0040] The external graphics memory employs a dual-buffering mechanism, divided into drawing data buffers A and B. When the graphics rendering module writes all the drawing data of a frame into drawing data buffer A, if the display output module is currently reading drawing data buffer B, it will wait until the display output module has finished reading drawing data buffer B before starting to write the next frame of drawing data into drawing data buffer B. Otherwise, the graphics rendering module directly writes the next frame of drawing data into drawing data buffer B. When the display output module has finished reading drawing data buffer A, if the graphics rendering module is currently writing to drawing data buffer B, the display output module will continue reading drawing data buffer A; otherwise, the display output module will read drawing data buffer B. This mechanism ensures the integrity of each frame of drawing data.

[0041] The display output module reads the primitive drawing data from the external graphics memory and outputs the display screen according to the VESA timing of the preset resolution.

[0042] The video synchronization module synchronizes the video data from the external EVS video clock domain to the clock domain of the display output module, which is used for superimposing the output of the display output module and the EVS video; the video synchronization module integrates distortion correction function.

[0043] Once the EVS video and display output modules are synchronized, image overlay output is achieved; it is possible to select to output EVS video separately, output the display output module's screen separately, or output an overlay of both images.

[0044] The image overlay is achieved by superimposing the pixel data of the EVS video and the output screen of the display output module. The formula is C=(D+E)-D×E / 255; where: D is the pixel value of the EVS video, E is the pixel value of the output screen of the display output module, and C is the pixel value of the overlaid image. Specific implementation examples:

[0046] A highly integrated graphics generation and distortion processing device in a head-up display system, the overall system logic architecture is as follows: Figure 1 As shown, it includes a drawing instruction cache module, a graphics drawing module, a cache module, a display output module, a video synchronization module, and an image overlay module. This graphics generation and distortion processing device not only has graphics generation functions, but also integrates image distortion correction and image overlay functions.

[0047] The drawing instruction cache module receives drawing instructions sent by the graphics rendering processor through an external bus. The bus between the graphics rendering processor and the graphics generation and distortion processing device is generally a high-speed bus such as PCIe or AXI.

[0048] Drawing commands generally include setting commands and drawing commands. Setting commands include parameter settings such as graphics resolution, window opening, and occlusion area; drawing commands include drawing commands for primitive types such as points, lines, circles, triangles, arcs, characters, and Bézier curves, as well as primitive drawing parameters such as color and line width.

[0049] After sending each frame of drawing instructions, the graphics rendering processor sends a buffer switching flag and stores it in the drawing instruction cache module. The drawing instruction cache module can cache a maximum of two frames of drawing instructions. The sending of drawing instructions by the graphics rendering processor and the reading of drawing instructions by the graphics rendering module are asynchronous. When the graphics rendering processor sends drawing instructions faster than the graphics rendering module reads them slower, two buffer switching flags will exist simultaneously in the drawing instruction cache module. In this case, the drawing instruction cache module will not accept new frames of drawing instructions until the graphics rendering module reads one of the buffer switching flags. Conversely, when the graphics rendering processor sends drawing instructions slower than the graphics rendering module reads them faster, the address of the drawing instructions read by the graphics rendering module will catch up with the address of the drawing instructions sent by the graphics rendering processor. At this point, the graphics rendering module stops drawing and waits for the graphics rendering processor to send new drawing instructions. This mechanism ensures the integrity of each frame of drawing instructions.

[0050] The graphics drawing module reads drawing instructions from the drawing instruction cache module, completes the setting of graphics parameters such as graphics resolution, window opening, and occlusion area, and then completes the drawing of basic graphic elements such as points, lines, circles, triangles, arcs, characters, and Bézier curves according to the graphics commands.

[0051] The display image output by the graphics generation and distortion correction device needs to be displayed through a head-up display (HUD), which contains a series of optical systems. If the display image is not processed in any way, the image perceived by the human eye will be severely distorted after passing through the optical system. Therefore, distortion correction processing is required to correct the distortion of the display image according to the distortion characteristics of the optical system.

[0052] The traditional distortion correction method based on FPGA has the following drawbacks: 1. It not only occupies the internal Block RAM resources of the FPGA, but also occupies FPGA pin resources to connect to external memory. 2. The distortion correction process has a millisecond-level delay. Therefore, a new distortion correction method is proposed in the graphics rendering module. Based on the distortion characteristics of the optical system, a fitting equation B = f(A) is generated. The graphics rendering module generates the original pixel coordinates (xa, ya) and color value (color). After calculation using the distortion fitting equation, the distorted pixel coordinates (xb, yb) are obtained. Since the coordinates (xb, yb) are decimals, the nearest integer coordinates (xc, yc) are found using the least nearest neighbor method. Finally, the graphics rendering module stores the distorted pixel coordinates (xc, yc) and pixel value (color) into external graphics memory.

[0053] The graphics drawing module stores the color value into the corresponding address of the external graphics memory based on the pixel coordinates of the generated drawing data. Before storing the color value, it is necessary to read the color value in the current external graphics memory address. The two color values ​​are mixed according to the coefficient K and then written into the external graphics memory. This can avoid the phenomenon that the lower color value written later at the same address will overwrite the higher color value written earlier, which will cause defects in the display screen.

[0054] After adding the readback mechanism, a cache module is added between the two to improve the efficiency of the graphics drawing module accessing external graphics memory.

[0055] The external graphics memory employs a dual-buffering mechanism, divided into drawing data buffers A and B. When the graphics rendering module writes all the drawing data for a frame into drawing data buffer A, if the display output module is currently reading from drawing data buffer B, it will wait until the display output module has finished reading from buffer B before starting to write the next frame of drawing data into buffer B. Otherwise, the graphics rendering module directly writes the next frame of drawing data into buffer B. When the display output module has finished reading from drawing data buffer A, if the graphics rendering module is currently writing to buffer B, it will continue reading from drawing data buffer A; otherwise, it will read from drawing data buffer B. This mechanism ensures the integrity of each frame of drawing data.

[0056] The display output module reads the drawing data from the external graphics memory and outputs the display screen according to the VESA timing set for the resolution.

[0057] The video synchronization module synchronizes video data from the external EVS video clock domain to the clock domain of the display output module, facilitating subsequent overlay processing of the two images. The video synchronization module also integrates distortion correction functionality, implemented in the same way as the distortion correction functionality integrated in the graphics rendering module.

[0058] Once the EVS video and the display output module output images are synchronized, image overlay can be achieved. Here, you can choose to output the EVS video, the image output by the display output module, or an overlay of both images.

[0059] Image overlay is achieved by superimposing pixel-by-pixel data between the EVS video and the output image from the display output module, using the formula C = (D + E) - D × E / 255. Where: D is the pixel value of the EVS video, E is the pixel value of the output image from the display output module, and C is the pixel value of the overlaid image.

Claims

1. An integrated graphics generation and distortion processing device for a head-up display system, characterized in that, It includes a drawing instruction cache module, a graphics drawing module, a cache module, a display output module, a video synchronization module, and an image overlay module; The drawing instruction cache module receives drawing instructions sent by the graphics rendering processor via an external bus; The graphics drawing module reads drawing instructions from the drawing instruction cache module, completes the setting of graphics parameters such as graphics resolution, window opening, and occlusion area, and then completes the drawing of primitives according to the graphics commands; the graphics drawing module integrates distortion correction function; The graphics drawing module stores the color value into the corresponding address of the external graphics memory based on the coordinates of the pixel points of the generated primitive drawing data. Before storing the color value, it reads the color value in the current external graphics memory address, and then writes the two color values ​​into the external graphics memory after weighted mixing. A cache module is added between the graphics drawing module and the graphics memory to improve the efficiency of the graphics drawing module in accessing the external graphics memory. The display output module reads the primitive drawing data from the external graphics memory and outputs the display screen according to the VESA timing of the preset resolution. The video synchronization module synchronizes the video data from the external EVS video clock domain to the clock domain of the display output module, which is used for superimposing the output of the display output module and the EVS video; the video synchronization module integrates distortion correction function. Once the output images from the EVS video and display output modules are synchronized, image overlay output is achieved. The interaction process between the graphics rendering processor, the drawing instruction cache module, and the graphics rendering module is as follows: After the graphics rendering processor sends out a frame of drawing instructions, it sends a switch buffer flag and stores it in the drawing instruction buffer module. The drawing instruction buffer module can buffer a maximum of two frames of drawing instructions. The graphics rendering processor sends drawing commands and the graphics rendering module reads drawing commands asynchronously. When the graphics rendering processor sends drawing commands quickly and the graphics rendering module reads drawing commands from the drawing command cache module slowly, there will be two switching cache flags in the drawing command cache module at the same time. At this time, the drawing command cache module will no longer receive the new frame of drawing commands until the graphics rendering module reads out one of the switching cache flags, and then the drawing command cache module will accept the new frame of drawing commands. When the graphics rendering processor sends drawing instructions slowly, but the graphics rendering module reads drawing instructions quickly, the address of the drawing instructions read by the graphics rendering module will catch up with the address of the drawing instructions sent by the graphics rendering processor. At this time, the graphics rendering module stops drawing and waits for the graphics rendering processor to send new drawing instructions. The image overlay is achieved by superimposing the pixel data of the EVS video and the output screen of the display output module, using the formula C=(D+E)-D×E / 255; where: D is the pixel value of the EVS video, E is the pixel value of the output screen of the display output module, and C is the pixel value of the overlaid image.

2. The integrated graphics generation and distortion processing device in a head-up display system according to claim 1, characterized in that, The external bus is a PCIE or AXI bus.

3. The integrated graphics generation and distortion processing device in a head-up display system according to claim 1, characterized in that, The graphics commands include drawing instructions for point, line, circle, triangle, arc, character, and Bézier curve primitive types, as well as drawing parameters for color and line width primitives.

4. The integrated graphics generation and distortion processing device in a head-up display system according to claim 1, characterized in that, The distortion correction function specifically includes: Based on the distortion characteristics of the optical system, a fitting equation formula B=f(A) is generated. The drawing data generated by the graphics rendering module consists of the original pixel coordinates (xa, ya) and color value color. After calculation by the distortion fitting equation formula, the distorted pixel coordinates (xb, yb) are obtained. Since the coordinates (xb, yb) are decimals, the nearest integer coordinates (xc, yc) are found according to the least nearest neighbor method. Finally, the graphics rendering module stores the distorted pixel coordinates (xc, yc) and pixel value color into the external graphics memory.

5. The integrated graphics generation and distortion processing device in a head-up display system according to claim 1, characterized in that, The external graphics memory employs a dual-buffering mechanism, divided into drawing data buffers A and B. When the graphics rendering module writes all the drawing data of a frame into drawing data buffer A, if the display output module is currently reading drawing data buffer B, it will wait until the display output module finishes reading drawing data buffer B before starting to write the next frame of drawing data into buffer B. Otherwise, the graphics rendering module will directly write the next frame of drawing data into buffer B. When the display output module finishes reading drawing data buffer A, if the graphics rendering module is currently writing into buffer B, it will continue reading drawing data buffer A; otherwise, it will read drawing data buffer B. This mechanism ensures the integrity of each frame of drawing data.

6. The integrated graphics generation and distortion processing device in a head-up display system according to claim 1, characterized in that, The image overlay output can select to output EVS video separately, output the screen of the display output module separately, or output an overlay of both images.

Images