Digital image generation and transmission device and system

Abstract

The embodiment of the invention discloses a digital image generation and transmission device and system. In one specific embodiment, the device comprises a display control unit, a graph generator, a frame memory controller and an output controller, the display control unit is used for generating a graphic calculation instruction according to a user instruction; the graph generator is used for calculating and generating a corresponding image according to the graph calculation instruction; the frame storage controller is used for storing the image as a whole frame of picture; and the output controller is used for reading the whole frame of picture stored in the frame storage controller and outputting two paths of videos in a differential format to external display equipment. According to the invention, a digital image generation and transmission device is designed based on a co-simulation image real-time generation technology of the FPGA, and the device can carry out the design of a multi-entity collaborative image generation task under a distributed collaborative condition, and can reasonably divide the calculation capability.

Description

Technical Field

[0001] The present invention relates to the technical field of semi-physical simulation, and more specifically, to a digital image generation and transmission device and system. Background Art

[0002] At present, in the simulation and verification of complex combat scenarios, it is necessary to use multi-channel digital image generation and transmission to transform the physical information of the same scene according to different observation angles, distances and other parameters to form digital image information for multi-entity collaborative detection. Digital image generation is a key link in the optical guidance and control semi-physical simulation system. This module must not only realize the real-time dynamic simulation of the geometric characteristics, optical characteristics, posture motion characteristics and relative distance change characteristics of the target group, but also realize the generation of digital images of complex combat collaborative scenes that are real-time controllable, stably transmitted, and consistent in time and space. At present, the data image generation and transmission mechanism based on commercial GPU has the problem that the update rate is not controlled by the exposure signal of the seeker, and the image generation and transmission time fluctuates, which will lead to the inconsistency of time and space in the characteristic elements of the target scene shared by multiple detection devices, and cannot meet the simulation and verification requirements of complex combat scenarios such as multi-entity collaborative detection.

[0003] In the era of "artificial intelligence +", military combat scenarios have gradually changed from small-scale single-arms scenarios and single-weapon combat to complex combat scenarios such as "land, sea, air, space and electricity" mixed scenarios, missile group coordination / drone cluster combat, which puts forward higher requirements on the cooperative guidance and control system of weapon systems. Scenario generation is an important means of guidance and control design verification, and multi-entity collaborative image generation has become the key and foundation of the development process of weapon guidance systems. In order to meet the needs of missile group coordination / drone cluster networked combat in complex combat scenarios, and adapt to the characteristics of complex multi-body collaborative system architecture, variable interconnection topology, and high spatiotemporal correlation of collaborative images, it is necessary to break through the limitations of traditional single weapon platform real-time simulation design, carry out research on a new generation of multi-entity collaborative image generation technology, carry out parallel real-time synchronous image generation, image data generation and transmission integration, and collaborative digital image synchronous generation and transmission methods, solve the problem of multi-user and multi-node collaborative real-time image generation in complex scenarios, and support the design and verification of collaborative guidance and control systems. Summary of the invention

[0004] An object of the present invention is to provide a digital image generation and transmission device and system to solve at least one of the problems existing in the prior art.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A first aspect of the present invention provides a digital image generation and transmission device, comprising:

[0007] Display control unit, graphics generator, frame memory controller and output controller;

[0008] The display control unit is used to generate graphic calculation instructions according to user instructions;

[0009] The graphics generator is used to calculate and generate a corresponding image according to the graphics calculation instruction;

[0010] The frame storage controller is used to save the image as a whole frame;

[0011] The output controller is used to read the entire frame stored in the frame memory controller and output two channels of video in a differential format to an external display device.

[0012] Optionally, the device further comprises a graphic element library;

[0013] The graphic element library is used to store the graphic element dot matrix information required for image generation.

[0014] A clock unit is used to provide clock information for the digital image generation and transmission device.

[0015] Optionally, the graphic calculation instructions include drawing instructions and background generation instructions;

[0016] The graphic generator includes a graphic generation unit, a background generation unit, an overlay output unit and a cache unit; wherein,

[0017] The graphics generation unit is used to draw dynamic graphics according to the drawing instructions and the primitive dot matrix information;

[0018] The background generation unit is used to generate a static background according to the background generation instruction;

[0019] The superposition output unit is used to extract the dynamic graphics cached by the cache unit and the static background to generate an image by fusion and superposition;

[0020] The cache unit is used to cache the dynamic graphics, static background and images.

[0021] Optionally, the graphics generation unit includes

[0022] A graphics drawing module, used to calculate the pixel information of the graphics to be drawn by anti-aliasing algorithm according to the drawing instruction; and to draw dynamic graphics by point scanning according to the pixel information;

[0023] A graphics transformation module, used to move, scale, rotate, mirror or adjust the brightness of the dynamic graphics;

[0024] A primitive drawing module, used to generate primitives in dynamic graphics according to primitive dot matrix information;

[0025] The dynamic graphics include points, straight lines, circles, arcs, multi-line width triangles, filled triangles, filled rectangles, filled circles and filled primitives.

[0026] Optionally, the device further comprises a power supply unit for supplying power to other devices in the device.

[0027] Optionally, the frame memory controller includes

[0028] The first DDR is used to store the color information of the entire frame;

[0029] The second DDR is used to store the grayscale information of the entire frame.

[0030] Optionally, the output controller includes

[0031] transmission control module, a first transceiver module, a second transceiver module, an LVDS driver module and an XGA driver module,

[0032] The LVDS driver module and the XGA driver module are used to output two channels of video in a differential format according to the stored full-frame images;

[0033] The transmission control module is used to extract the entire frame image stored in the cache unit according to the time information and transmit it to the LVDS driver module and the XGA driver module; it is also used to select the first transceiver interface or the second transceiver interface to output two-way video in differential format to an external display device according to the selection instruction in the user instruction.

[0034] Optionally, the device further comprises

[0035] Optical fiber interface, used to receive user instructions provided externally.

[0036] Optionally, the device further comprises

[0037] Gigabit network port, used to receive time synchronization pulses transmitted by external devices;

[0038] A time synchronization module is used to calculate the time deviation between the digital image generation and transmission device and an external device according to the time synchronization pulse, and adjust the clock unit according to the time deviation.

[0039] A second aspect of the present invention provides a digital image generation and transmission system using a digital image generation and transmission device, the system comprising a plurality of digital image generation and transmission devices and a gigabit network switch;

[0040] The digital image generation and transmission device is used to output synchronization pulses;

[0041] The Gigabit network switch is used to forward the output synchronization pulse to the time synchronization module of the digital image generation and transmission device, so as to synchronize the time of multiple digital image generation and transmission devices.

[0042] The beneficial effects of the present invention are as follows:

[0043] The present invention designs a digital image generation and transmission device based on FPGA collaborative simulation image real-time generation technology, which can design multi-entity collaborative image generation tasks under distributed collaborative conditions and can reasonably divide computing power. The present invention also provides support for multiple simulation image scenes through dynamic loading of graphic generation units or graphic transformation modules to achieve efficient collaborative image generation. The present invention can also achieve time synchronization between devices through a time synchronization module to achieve collaborative output of videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0045] Figure 1 A schematic diagram showing a digital image generating and transmitting device in one embodiment of the present invention.

[0046] Figure 2 A schematic diagram showing an output controller in one embodiment of the present invention.

[0047] Figure 3 A block diagram showing time synchronization of distributed image generation of multiple digital image generation and transmission devices in one embodiment of the present invention is shown.

[0048] Figure 4 A schematic diagram showing a digital image generation and transmission system in one embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to more clearly illustrate the present invention, the following embodiments and accompanying drawings are provided. Figure 1-4 The present invention is further described. Similar components in the drawings are indicated by the same reference numerals. Those skilled in the art should understand that the following specific description is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present invention.

[0050] like Figure 1 As shown, a digital image generation and transmission device comprises:

[0051] Display control unit, graphics generator, frame memory controller and output controller;

[0052] The display control unit is used to generate graphic calculation instructions according to user instructions;

[0053] The graphics generator is used to calculate and generate a corresponding image according to the graphics calculation instruction;

[0054] The frame storage controller is used to save the image as a whole frame;

[0055] The output controller is used to read the entire frame stored in the frame memory controller and output two channels of video in a differential format to an external display device.

[0056] Specifically, the digital image generation and transmission device is deployed on the FPGA board, and the board is inserted into the PCIe expansion slot of the industrial computer, which is plug-and-play and convenient for subsequent expansion into multiple FPGA joint simulations; the core FPGA processor communicates with the industrial computer through the PCIe bus;

[0057] The present invention designs a digital image generation and transmission device based on FPGA-based collaborative simulation image real-time generation technology. The device can design multi-entity collaborative image generation tasks under distributed collaborative conditions and can reasonably divide computing power.

[0058] In one embodiment, the apparatus further comprises a graphic primitive library;

[0059] The graphic element library is used to store the graphic element dot matrix information required for image generation.

[0060] A clock unit is used to provide clock information for the digital image generation and transmission device.

[0061] Specifically, the primitive library is implemented with EMMC storage chip to store the primitive dot matrix information required for image generation. When characters need to be displayed, FPGA will access EMMC to read the primitive information. The clock unit is designed to take into account the timing control of XGA drive, and the external input clock is designed to be 65MHz.

[0062] In one embodiment, the graphics calculation instructions include drawing instructions and background generation instructions;

[0063] The graphic generator includes a graphic generation unit, a background generation unit, an overlay output unit and a cache unit; wherein,

[0064] The graphics generation unit is used to draw dynamic graphics according to the drawing instructions and the primitive dot matrix information;

[0065] The background generation unit is used to generate a static background according to the background generation instruction;

[0066] The superposition output unit is used to extract the dynamic graphics cached by the cache unit and the static background to generate an image by fusion and superposition;

[0067] The cache unit is used to cache the dynamic graphics, static background and images.

[0068] Specifically, the graphic data is directly stored in the cache unit without processing, and is superimposed and output after background fusion processing with the background data. After the dynamic graphics are generated, the data is directly stored in the graphics cache. When fusion is performed, the graphic grayscale value and interpolation coefficient are read from the graphics cache, and the background grayscale value is read from the background cache. The background fusion calculation formula is selected according to the highest bit of the interpolation coefficient corresponding to each point, and the superimposed output is directly performed after the calculation is completed.

[0069] In one embodiment, the graphics generation unit includes

[0070] A graphics drawing module, used to calculate the pixel information of the graphics to be drawn by anti-aliasing algorithm according to the drawing instruction; and to draw dynamic graphics by point scanning according to the pixel information;

[0071] A graphics transformation module, used to move, scale, rotate, mirror or adjust the brightness of the dynamic graphics;

[0072] A primitive drawing module, used to generate primitives in dynamic graphics according to primitive dot matrix information;

[0073] The dynamic graphics include points, straight lines, circles, arcs, multi-line width triangles, filled triangles, filled rectangles, filled circles and filled primitives.

[0074] The present invention also provides support for multiple simulation image scenes through dynamic loading of a graphic generation unit or a graphic transformation module, thereby achieving efficient collaborative image generation.

[0075] In one embodiment, the apparatus further comprises a power supply unit for supplying power to other devices in the apparatus.

[0076] In one embodiment, the frame memory controller comprises

[0077] The first DDR is used to store the color information of the entire frame;

[0078] The second DDR is used to store the grayscale information of the entire frame.

[0079] Specifically, when the digital image generation and transmission device is working, the display control unit generates graphic calculation instructions according to user instructions, the graphic generation unit reads and decodes the graphic calculation instructions, and generates corresponding graphics according to different graphic calculation instructions. The calculated graphic pixel information is saved by the frame memory controller to the DDR (Double Data Rate SDRAM, double-speed synchronous dynamic random access memory) group (first DDR and second DDR), and the frame memory controller reads the complete frame picture pixel information saved in the DDR group for output. Due to the parallel computing characteristics of FPGA, the image output can simultaneously calculate a new frame picture and control other functional parts, thereby ensuring the real-time and continuity of the graphic picture output.

[0080] In one embodiment, Figure 1 As shown, the output controller includes

[0081] LVDS driver module and XGA driver module,

[0082] The LVDS driver module and the XGA driver module are used to output two channels of video in a differential format according to the stored full-frame images;

[0083] like Figure 2 As shown, the output controller also includes a transmission control module, a first transceiver module and a second transceiver module

[0084] The transmission control module is used to extract the entire frame image stored in the cache unit according to the time information and transmit it to the LVDS driver module and the XGA driver module; it is also used to select the first transceiver interface or the second transceiver interface to output two-way video in differential format to an external display device according to the selection instruction in the user instruction.

[0085] Specifically, the present invention adopts selectIO (optional IO) and gtwizard (high-speed transceiver) IP cores provided by Xilinx to realize a multi-mode image transmission interface, and realizes data flow control between image buffer and configurable SelectIO interface (first transceiver interface) and configurable Gtwizard interface (second transceiver interface) through a transmission control unit, thereby realizing a flexible and configurable multi-mode and multi-category image transmission interface.

[0086] The common ways of image output are classified into two categories: high-speed serial bus based on LVDS and high-speed transceiver bus based on LVDS. Choose a channel to realize image data transmission according to the task requirements.

[0087] The commonly used image transmission interfaces in the simulation field can be divided into two categories: one is a high-speed serial bus based on LVDS level, such as parallel multi-channel LVDS, Camelink interface, etc.; the other is a high-speed serial interface based on a high-speed transceiver interface, such as CXP interface, fiber Aurora protocol interface, SRIO interface, etc. In the design, selectIO (optional IO) and gtwizard (high-speed transceiver) provided by Xilinx are used. From the perspective of generalization, the IP core unifies the two typical interfaces internally and forms a flexible multi-mode image transmission interface with external multi-mode. Among them, SERDESE2 in SelectIO is a dedicated serial-to-parallel converter. It does not bring unnecessary timing problems when completing the serial-to-parallel conversion, so it is very suitable for application in high-speed source synchronous applications. For example, image data. Dedicated deserializer / serial-to-parallel converter; it can complete high-speed data transmission without the need for the FPGA end to match the data frequency, and supports SDR (single data rate) and DDR (double data rate) modes. SDR mode supports 2-, 3-, 4-, 5-, 6-, 7-, 8-bit bit width; DDR mode supports 4-, 6-, 8-bit bit width. 10 or 14-bit requires two cascades. Bitslip submodule can be used to rearrange the input parallel data and can be used for source synchronous tranining.

[0088] Through SelectIO control, various LVDS digital image outputs can be realized based on the LVDS interface.

[0089] GTWiazrd transceiver uses differential signal to transmit data, and the commonly used differential signal standard is CML (current mode logic). It mainly includes PMA and PCS:

[0090] The main functions of PMA (Physical Medium Attachment) are serial-to-parallel / parallel-to-serial conversion and analog part. It provides high-performance serial interface, such as pre-emphasis and equalization. PCS (Physical Coding Sublayer) is mainly parallel digital circuit processing. Encoding / decoding of streaming data, 8B\10B, etc. GTWiazrd can realize high-speed serial interface based on serdes (serializer), and improve data transmission reliability through pre-emphasis and equalization.

[0091] In one embodiment, the device further comprises

[0092] Optical fiber interface, used to receive user instructions provided externally.

[0093] Optionally, the device further comprises

[0094] Gigabit network port, used to receive time synchronization pulses transmitted by external devices;

[0095] A time synchronization module is used to calculate the time deviation between the digital image generation and transmission device and an external device according to the time synchronization pulse, and adjust the clock unit according to the time deviation.

[0096] Specifically, the time synchronization module calculates the time deviation between the digital image generation and transmission device and the external device according to the time synchronization algorithm and the time synchronization pulse, and adjusts the clock unit according to the time deviation. The designed time synchronization algorithm solves the problems of clock phase alignment and clock drift through delay measurement, clock correction and frequency drift compensation design methods; network time synchronization in this technology is an improvement of the existing technology, and the synchronization principle draws on the synchronization principle in the IEEE1588 protocol. IEEE1588 defines a precise synchronization clock protocol (PTP protocol) in industrial automation systems, which is related to network communication, local computing and distribution objects. The IEEE1588 clock protocol stipulates that when performing clock synchronization, the master device first sends a clock synchronization message in the form of multicast, and all devices in the same domain as the master device will receive the synchronization message. After receiving the synchronization message, the slave device calculates the deviation from the master clock according to the timestamp in the synchronization message and the line delay from the master clock to the slave clock, and adjusts the local clock. The digital image generation and transmission device uses hard-wired time synchronization, and multiple digital image generation and transmission devices use a combination of hard-wired and networked time synchronization. The cache unit includes N+M cache frames, and N>2, for example Figure 3 The cache frames 2-1, 2-2, 2-3, ..., NM are shown; the cache frames are used to cache images; Figure 3 As shown, the images generated by the cache management are synchronously output to different cache frames, and the time synchronization modules of multiple digital image generation and transmission devices use the Class 1588 time synchronization protocol to maintain synchronized timestamps in multiple images output by the cache unit to achieve collaborative image time calibration.

[0097] Specifically, the graphic data is directly stored in the cache unit without processing, and is superimposed and output after background fusion processing with the background data. After the dynamic graphics are generated, the data is directly stored in the graphics cache. When fusion is performed, the graphic grayscale value and interpolation coefficient are read from the graphics cache, and the background grayscale value is read from the background cache. The background fusion calculation formula is selected according to the highest bit of the interpolation coefficient corresponding to each point, and the superimposed output is directly performed after the calculation is completed.

[0098] In one embodiment, Figure 4 As shown, a digital image generation and transmission system using a digital image generation and transmission device, the system includes a plurality of digital image generation and transmission devices and a gigabit network switch;

[0099] The digital image generation and transmission device is used to output synchronization pulses;

[0100] The Gigabit network switch is used to forward the output synchronization pulse to the time synchronization module of the digital image generation and transmission device, so as to synchronize the time of multiple digital image generation and transmission devices.

[0101] Specifically, the digital image generation and transmission device can achieve distributed expansion through the Gigabit network port, so that multi-channel images can be realized in complex scenes. If collaborative matching between images is required, the synchronization mechanism of the image generation device can also be realized under the network interface. When multiple image generation devices generate collaborative images, they first complete network time synchronization through Ethernet connection; and maintain synchronization timestamps in each digital image generation and transmission device. After that, the single-board synchronization signal can be generated through the delay line to achieve collaborative image time calibration, thereby realizing collaborative image generation.

[0102] The present invention designs a digital image generation and transmission device based on FPGA collaborative simulation image real-time generation technology, which can design multi-entity collaborative image generation tasks under distributed collaborative conditions and can reasonably divide computing power. The present invention also provides support for multiple simulation image scenes through dynamic loading of graphic generation units or graphic transformation modules to achieve efficient collaborative image generation. The present invention can also achieve time synchronization between devices through a time synchronization module to achieve collaborative output of videos.

[0103] In the description of the present invention, it should be noted that the terms "upper", "lower", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific position, be constructed and operated in a specific position, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0104] It should also be noted that, in the description of the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0105] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the embodiments here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A digital image generation and transmission device, characterized in that: include: Display control unit, graphics generator, frame memory controller and output controller; The display control unit is used to generate graphic calculation instructions according to user instructions; The graphics generator is used to calculate and generate a corresponding image according to the graphics calculation instruction; The frame storage controller is used to save the image as a whole frame; The output controller is used to read the entire frame stored in the frame memory controller and output two channels of video in a differential format to an external display device.

2. The device according to claim 1, characterized in that The device also includes a graphic element library; The graphic element library is used to store graphic element dot matrix information required for image generation; A clock unit is used to provide clock information for the digital image generation and transmission device.

3. The device according to claim 2, characterized in that The graphic calculation instructions include drawing instructions and background generation instructions; The graphic generator includes a graphic generation unit, a background generation unit, an overlay output unit and a cache unit; wherein, The graphics generation unit is used to draw dynamic graphics according to the drawing instructions and the primitive dot matrix information; The background generation unit is used to generate a static background according to the background generation instruction; The superposition output unit is used to extract the dynamic graphics cached by the cache unit and the static background to generate an image by fusion and superposition; The cache unit is used to cache the dynamic graphics, static background and images.

4. The device according to claim 3, characterized in that The graphics generation unit includes A graphics drawing module, used to calculate pixel information of the graphics to be drawn by using an anti-aliasing algorithm according to the drawing instruction; It also draws dynamic graphics by point scanning based on pixel information; A graphics transformation module, used to move, scale, rotate, mirror or adjust the brightness of the dynamic graphics; A primitive drawing module, used to generate primitives in dynamic graphics according to primitive dot matrix information; The dynamic graphics include points, straight lines, circles, arcs, multi-line width triangles, filled triangles, filled rectangles, filled circles and filled primitives.

5. The device according to claim 4, characterized in that The device further comprises a power supply unit for supplying power to other devices in the device.

6. The device according to claim 5, characterized in that The frame memory controller comprises The first DDR is used to store the color information of the entire frame; The second DDR is used to store the grayscale information of the entire frame.

7. The device according to claim 6, characterized in that The output controller includes transmission control module, a first transceiver module, a second transceiver module, an LVDS driver module and an XGA driver module, The LVDS driver module and the XGA driver module are used to output two channels of video in a differential format according to the stored full-frame images; The transmission control module is used to extract the entire frame of the picture stored in the cache unit according to the time information and transmit it to the LVDS driver module and the XGA driver module; it is also used to select the first transceiver interface or the second transceiver interface to output two-way video in differential format to an external display device according to the selection instruction in the user instruction.

8. The device according to claim 7, characterized in that The device also includes Optical fiber interface, used to receive user instructions provided externally.

9. The device according to claim 8, characterized in that The device also includes Gigabit network port, used to receive time synchronization pulses transmitted by external devices; The time synchronization module is used to calculate the time deviation between the digital image generation and transmission device and the external device according to the time synchronization pulse, and adjust the clock unit according to the time deviation.

10. A digital image generation and transmission system using the digital image generation and transmission device according to any one of claims 1 to 9, characterized in that: The system includes a plurality of digital image generation and transmission devices and a gigabit network switch; The digital image generation and transmission device is used to output synchronization pulses; The Gigabit network switch is used to forward the output synchronization pulse to the time synchronization module of the digital image generation and transmission device, so as to synchronize the time of multiple digital image generation and transmission devices.

Images