A method and system for displaying image and video based on FPGA-mounted chip

Abstract

The application belongs to the technical field of image video, and discloses a display method and system for image video based on FPGA-mounted chips. An ARM processor receives a video display instruction periodically issued by an upper computer, generates an indication signal and a sub-block image data serial number for a display control processor FPGA, triggers the display control processor FPGA to sample, and stores the sub-block image data in a BRAM according to the serial number. After storing the pixel control information of a complete sub-block, the ARM processor is replied with a handshake signal, and the ARM processor is triggered to send the next sub-block image data until the display control processor FPGA receives and stores all the background area information of the video, triggers the image encoder state machine to work, and completes the display control task of different scene pictures through the encoder BT656 transmission protocol. The application makes the display control of the airborne equipment applicable to more different scenes, and the control is more flexible and the hardware cost is lower.

Description

Technical Field

[0001] This invention belongs to the field of image and video technology, and in particular relates to a method and system for displaying images and videos based on an FPGA-equipped chip. Background Technology

[0002] The video display of various airborne devices is crucial for the coordinated operation of these devices and for issuing commands to operators. Operators control the aircraft's speed and attitude based on background text and the movement of the central cursor in the video. The ADV7341 chip supports the BT656 image encoding protocol to complete the video display control function. The FPGA, acting as a processor, configures the 7341 via the I2C interface to achieve SD and PAL format video output, displaying the video at a frame rate of 25fps.

[0003] Based on the above analysis, the problems and defects of the existing technology are as follows: In the existing technology, the display control of airborne equipment has poor applicability in different scenarios, is cumbersome to control, and has high hardware costs. Summary of the Invention

[0004] To overcome the problems existing in related technologies, the present invention discloses an embodiment of a method and system for displaying images and videos based on an FPGA-mounted chip, specifically relating to the development of logic gate arrays in aerospace and other fields, particularly for the display control of images and videos on airborne equipment, and specifically providing a method for displaying and controlling images and videos based on an FPGA-mounted ADV7341 chip.

[0005] The technical solution is as follows: a method for displaying images and videos based on an FPGA-mounted chip, which runs on an SOPC chip, including an ARM processor and a display control processor FPGA;

[0006] The ARM processor receives video display instructions periodically issued by the host computer. The video display instructions include multiple sub-block image data of the video background image after the video background area is planned and divided.

[0007] The ARM processor adopts a block transmission method. Each time a sub-block of image data is transmitted, the display control processor FPGA generates a wr_done indicator signal and a sub-block image data sequence number, triggering the display control processor FPGA to sample and store the pixel control information of a complete sub-block in the BRAM according to the sequence number. The sequence number is the starting address of the BRAM.

[0008] After storing the pixel control information of each complete sub-block, the display control processor FPGA replies to the ARM processor with a handshake signal wr_ok and triggers the ARM processor to send the next sub-block image data. This handshake operation is continuously repeated until the display control processor FPGA has received and stored all the background area information of the video. Based on the background area information of all the received video, the display control processor FPGA triggers the image encoder state machine to work by configuring the timing of the I2C configuration of the image encoder ADV7341, and completes the display control task of different scene images through the encoder BT656 transmission protocol.

[0009] Furthermore, after receiving the video display command, the ARM processor completes the handshake with the display control processor FPGA and the periodic transmission of image data and control commands through the 32-bit FSMC32 bus;

[0010] The handshake operation includes: the ARM processor sending a wr_done signal to the display control processor FPGA, and the display control processor FPGA receiving the signal and replying with wr_ok, where 1 indicates that it is valid.

[0011] Furthermore, the ARM processor implements periodic transmission of the 32-bit FSMC32 bus by configuring frequency division, frequency multiplication, and selecting the enable register.

[0012] Furthermore, after the display control processor FPGA receives the background area information of all videos, it checks whether the image encoder ADV7341 has been configured. If it has been configured, it triggers the image encoder ADV7341 to work.

[0013] Furthermore, the video background area planning includes: the display content of a frame image is represented by the pixel black-and-white control information of a set area, and the area planning is represented by sub-blocks in four directions: upper left, upper right, lower left, and lower right. The pixel black-and-white control includes: 1 from the SOPC chip indicates that a specific pixel is lit, and 0 indicates that a specific pixel is not lit. The display control processor FPGA decodes 1 as: 8'h10 black lit, and decodes 0 as: 8'hf0 white default state.

[0014] Furthermore, the display control processor FPGA completes the register configuration of the internal working mode of the image encoder ADV7341 by simulating I2C communication timing. The specific configured registers include: chip reset mode control register, chip power supply mode control register, SD standard definition control register, YCbCr mode register, single-channel 8-bit encoder output control register, internal color bar test disable register, output selection register, subcarrier frequency control register, and PAL / NTSC format selection register.

[0015] The internal operating mode of the image encoder ADV7341 includes the generation of the encoder operating clock. The encoder operating clock is the operating clock for the image encoder ADV7341 to perform internal DAC conversion by image digital encoding. It is a 27MHz clock, which includes the CLOCK, HCLK and CLKOUT clock signals.

[0016] Furthermore, the image encoder ADV7341 uses synchronization code EAV, with SAV serving as synchronization information for encoder encoding.

[0017] Furthermore, the specific process of the encoder BT656 transmission protocol is as follows:

[0018] Step 1: The encoder state machine is in the IDLE state. Initialize the pixel pointer Pixels = 0 and the line pointer Lines = 0. The encoder data state defaults to the even field blanking end code state EAV. When START_ENCODING = 1, jump to step 2.

[0019] Step 2: The encoder is in WORK state, and the pixel pointer Pixels is incremented by 1 on the falling edge of HCLK. When the EAV state is (Pixels<=4), the preamble codes 8'hff, 8'h00, 8'h00, and EAVx are generated respectively. When the pixel pointer Pixels>4, jump to step 3.

[0020] Step 3: The encoder enters the line blanking area. When the pixel pointer satisfies 5 <= Pixels <= 283, it generates the padding code 8'h10 when the value of the Pixels pointer is odd and generates the padding code 8'h80 when the value of the Pixels pointer is even. When the pixel pointer satisfies Pixels >= 284, it jumps to step 4.

[0021] Step 4: The encoder enters the SAV state and, based on the current number of lines encoded, enters the even field blanking start SAV1, even field data start SAV2, odd field blanking start SAV4, and odd field data start SAV5 states respectively. Similar to step 2, it generates preambles: 8'hff, 8'h00, 8'h00, and SAVx. When the pixel indicator Pixels > 288, it jumps to step 5.

[0022] Step 5: The encoder enters the data encoding DATA state. According to the YCbCr422 encoding format, it sequentially enters the Y_DATA1 state, ODD state, Y_DATA2 state, and EVEN state. When it detects that the pixel pointer Pixels >= 1728, it increments the line index Lines by 1 and sets the pixel pointer Pixels to 1. It then checks whether the current line has reached MAXLINE = 625. If it has not, it jumps to step 2; otherwise, it jumps to step 6.

[0023] Step 6: The encoder has completed the refresh encoding of a complete frame and enters the END state. At this time, the encoded data status datadone is pulled high to indicate that the encoding has been completed. Then, it returns to Step 1 and waits for new data or commands to continuously refresh the screen.

[0024] Another objective of this invention is to provide an image and video display system based on an FPGA-mounted chip. This system is implemented using the aforementioned method for displaying images and videos based on an FPGA-mounted chip. The system includes:

[0025] The host computer is used to periodically send video display commands to the SOPC chip;

[0026] The SOPC chip is used to generate control information for the ADV7341 image encoder.

[0027] The ADV7341 image encoder integrates an I2C slave interface to acquire configuration data from the SOPC chip. It uses the BT656 encoder transmission protocol and works with a video driver to complete display control tasks for different scene images.

[0028] Furthermore, the SOPC chip includes an ARM processor and a display control processor (FPGA);

[0029] The ARM processor receives video display commands periodically issued by the host computer;

[0030] The ARM processor adopts a block transmission method. For each sub-block of image data transmitted, a wr_done indicator signal and a sub-block image data sequence number are generated for the display control processor FPGA. This triggers the display control processor FPGA to sample the data and store it in the BRAM according to the sequence number, thus storing the pixel control information of a complete sub-block.

[0031] Combining all the above technical solutions, the advantages and positive effects of this invention are as follows: This invention receives video display commands from a host computer in real time through an airborne display and control module. The airborne display and control module, according to the commands, controls the internal image encoder via the BT656 protocol to complete the display control task of specific scene images.

[0032] In this invention, the processor uses an XC2V1000-4fgg FPGA. A 100MHz clock is divided by 50 to 2MHz as the configuration clock for the I2C interface. At the same time, a 27MHz clock is used to implement the encoder protocol of BT656, thereby generating pixel control information and synchronous blanking control signals for ADV7341, so as to realize the speed indication, direction indication, and wide and narrow field of view indication of the center cursor.

[0033] This invention proposes a display control method for images and videos based on an FPGA equipped with an ADV7341 chip, which makes the display control of airborne equipment applicable to more different scenarios, more flexible in control, and lower in hardware cost. Attached Figure Description

[0034] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure;

[0035] Figure 1 This is a schematic diagram of a method for displaying images and videos based on an FPGA-mounted chip, provided in an embodiment of the present invention.

[0036] Figure 2 This is a flowchart of a method for displaying images and videos based on an FPGA-mounted chip, provided in an embodiment of the present invention.

[0037] Figure 3 This is a flowchart of the encoder BT656 transmission protocol provided in an embodiment of the present invention. Detailed Implementation

[0038] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0039] The innovation of the image and video display method and system based on FPGA chip provided in this invention lies in the following: This invention uses a low-end FPGA and optimizes internal hardware resources to achieve relatively ideal screen switching and display control with good real-time performance. Furthermore, it innovatively proposes an image segmentation and stitching storage and processing method. This invention is the first to use an FPGA, rather than an ASIC or ARM processor, to achieve flexible video screen display control.

[0040] Example 1: The method for displaying images and videos based on an FPGA-mounted chip provided in this embodiment of the invention includes:

[0041] It runs on the SM9B100BAL SOPC chip, which includes an ARM processor and a 1 million gate FPGA (compatible with imported XC2V1000-4ffg);

[0042] The ARM processor receives video display instructions periodically issued by the host computer. The video display instructions include multiple sub-block image data of the video background image after the video background area is planned and divided.

[0043] The ARM processor adopts a block transmission method. For each sub-block of image data transmitted, a wr_done indicator signal and a sub-block image data sequence number are generated for the display control processor FPGA. This triggers the display control processor FPGA to sample the data and store it in the BRAM according to the sequence number. This stores the pixel control information of a complete sub-block. The sequence number is the starting address of the BRAM.

[0044] After storing the pixel control information of each complete sub-block, the display control processor FPGA replies to the ARM processor with a handshake signal wr_ok and triggers the ARM processor to send the next sub-block image data. This handshake operation is continuously repeated until the display control processor FPGA has received and stored all the background area information of the video. Based on the background area information of all the received video, the display control processor FPGA triggers the image encoder state machine to work by configuring the timing of the I2C configuration of the image encoder ADV7341, and completes the display control task of different scene images through the encoder BT656 transmission protocol.

[0045] For example, the handshake operation includes: the ARM processor sending a wr_done signal to the FPGA, and the FPGA receiving the signal and replying with wr_ok, where 1 indicates that it is valid.

[0046] For example, receiving video data and video display instructions includes:

[0047] The video display command is sent from the host computer to the main control module via the 1553B bus. The main control module includes an ARM processor. After receiving the video display command, the ARM processor completes the handshake with the display control processor FPGA and the periodic transmission of image data and control commands through the FSMC32 bus. The ARM processor's main frequency is set to 144MHz. By configuring registers such as frequency divider, frequency multiplier, and enable selection, the FSMC32 bus cycle is made 220ns. The display control processor FPGA acts as the receiving end, using a 100MHz clock to reliably sample the data on the FSMC32 bus to complete the reception of image data and video display commands.

[0048] In this embodiment of the invention, the video background image adopts a segmented display strategy. After the ARM processor sends the image data of a sub-block, it generates a wr_done indication signal to the display control processor FPGA, triggering the display control processor FPGA to sample the image data on the bus. After the display control processor FPGA receives and stores the pixel control information of a complete sub-block in its internal BRAM, it replies to the ARM processor with a handshake signal wr_ok, triggering the ARM processor to send the image data of the next sub-block. The handshake operation is continuously repeated until the display control processor FPGA has received and stored all the background information of the video, and then sends an all_wr_ok handshake signal back to the ARM processor, notifying the ARM processor that it can send display control commands, and at the same time triggering the operation of the BT656 encoder state machine.

[0049] In this embodiment of the invention, the video background image is displayed in blocks, and the video background area planning includes:

[0050] The aforementioned video background refers to the on-screen operation instructions, which, in conjunction with the various movements of the central cursor, guide the operator to identify the current state of the aircraft and the subsequent operation instructions. The aforementioned area planning refers to the representation of the display content of a frame of image by the pixel black-and-white control information of a defined area. This invention uses sub-blocks in four directions—upper left, upper right, lower left, and lower right—to represent the area planning, with each sub-block having a resolution of 32*32 pixels. Regarding pixel black-and-white control, it is agreed that "1" from the main control module indicates that a specific pixel is lit, and "0" indicates that a specific pixel is not lit. The FPGA decodes "1" as 8'h10 (black lit) and "0" as 8'hf0 (white default state).

[0051] In this embodiment of the invention, the configuration of the image encoder ADV7341 includes:

[0052] Encoder configuration refers to the process by which the FPGA, the display control processor, configures the registers of the image encoding chip's internal operating mode through simulated I2C communication timing. Specifically, the registers that need to be configured are: 0x17 (chip reset mode control register), 0x00 (chip power supply mode control register), 0x01 (SD standard definition control register), 0x87 (YCbCr mode register), 0x88 (single-channel 8-bit encoder output control register), 0x84 (internal color bar test disable register), 0x82 (output selection register), 0x8C (subcarrier frequency control register), and 0x80 (PAL / NTSC format selection register). The remaining registers can be set to their default settings according to the manual, or they do not need to be configured separately.

[0053] In this embodiment of the invention, the generation of the encoder operating clock includes:

[0054] The encoder working clock refers to the working clock of the image encoder ADV7341 for performing DAC conversion by image digital encoding. It is fixed at 27MHz. However, this 27MHz clock needs to be generated by the display control processor FPGA in order to synchronize with the operation of the display control processor FPGA. In this invention, the 27MHz clock includes three different clock signals: CLOCK, HCLK and CLKOUT.

[0055] CLOCK is the input clock of the display control processor FPGA through the global clock buffer IBUFG, and its clock source is an external 27M crystal oscillator with a certain error; HCLK is the synchronous master clock of the display control processor FPGA to implement the encoder state machine; CLKOUT is the synchronous slave clock generated by the display control processor FPGA and provided to the image encoder ADV7341.

[0056] CLKOUT is the phase-shifted clock of HCLK, with its phase shifted by two clock cycles lagging HCLK relative to the CLOCK clock. Since the sensitive reference signal is the input clock CLOCK, CLKOUT cannot be obtained simply by pausing HCLK, as this would lead to timing misalignment and garbled images. The phase shift is generated using a gated clock method. This gated clock refers to releasing CLOCK at a specific moment to achieve timing alignment. This ensures that the encoded data generated by the FPGA referenced by HCLK is reliably sampled by the slave clock CLKOUT referenced by the ADV7341 image encoder, ensuring accurate control of pixel information.

[0057] In this embodiment of the invention, the BT656 protocol implemented based on a finite state machine includes:

[0058] The BT656 protocol refers to the FPGA (Field-Programmable Array Processor) transmitting timing information consistent with the image encoding protocol, enabling the ADV7341 image encoder to reliably receive the signals and achieve the desired encoding effect. For standard definition PAL format encoding, with a resolution of 720*576, a YCbCr (4:2:2) encoding format is required. Each pixel is described by two luminance components, Y1 and Y2, and a chrominance component, CbCr. Therefore, each line requires 1440 chrominance and luminance information. Including the time for line blanking, encoding one line of image requires a total of 1728 HCLK clock cycles. In this invention, the variable Pixels is used to count the cycle number of the current HCLK (main control 27MHz clock) within the current line, serving as the column index for the current encoding, completing the scan of one line (one line of data includes: 4clk of EAV synchronization code + 280clk of 8010 padding code + 4clk of SAV synchronization code + 1440clk of pixel data). The PAL format refers to an interlaced scanning method with 625 scan lines and a frame rate of 25fps. That is, a single frame consists of 625 scan lines: even field scan lines (288) + odd field scan lines (288) + field blanking lines (49). Therefore, in this invention, the variable Lines (maximum 625) is used as the line index for the current encoding. This invention uses the internal synchronization code EAV and SAV as the synchronization information encoded by the encoder, resulting in a simpler implementation and requiring fewer constrained signals.

[0059] Example 2, as another embodiment of the present invention, uses the SM9B100BAL SOPC chip as the display control processor FPGA chip. This SOPC chip integrates an ARM 926 core processor and a 1 million-gate logic gate array, is compatible with the XC2V1000 model display control processor FPGA, has a CPU frequency of up to 144MHz, and the on-chip ARM can communicate with the display control processor FPGA through the FSMC 32-bit internal bus, which greatly saves external IO resources and has strong scalability.

[0060] The I2C bus in this example conforms to the Philips standard and is a simple, bidirectional two-wire synchronous serial bus with SCL and SDA. Data is updated on SDA when SCL is low, and reliable sampling is possible during the high level of SCL. The interface circuit is an open-drain output, and both the master and slave devices have control over the I2C bus. A typical write operation involves three stages: (sending the destination device address - sending the device's internal register address - writing the register configuration). Each stage requires nine I2C clock cycles. In the ninth clock cycle of each stage, the master releases bus control (high impedance state) and switches it to the slave to generate an ACK response.

[0061] In this example, the display control processor FPGA sequentially provides configuration data for certain registers of the image encoder ADV7341 via the I2C bus.

[0062] The ADV7341 image encoder chip in this example is an encoder chip compatible with the BT656 transmission protocol. It integrates an I2C slave interface to obtain configuration data from the host to determine the operating mode. The ADV7341 integrates three 10-bit digital encoding input interfaces, supporting RGB, YUV422, and YUV444 formats. It also supports six analog output DAC interfaces and simultaneously supports NTSC and PAL video image output. The ADV7341 chip's output is typically paired with an SGM9111 video driver to enhance the driving capability of the DAC video signal output from the ADV7341 to the aviation connector.

[0063] As can be seen from the above embodiments, the present invention effectively reduces the cost of a single board, facilitates software and hardware maintenance, and has strong reusability.

[0064] This invention is the first to use an FPGA to implement the BT656 image transmission protocol and encoder algorithm. By rationally optimizing hardware resources and the encoder algorithm, this invention enables the display of relatively complex and flexible video images using only a low-end 2V1000 series FPGA. This invention overcomes the prejudice that video image processing generally requires high-end processors or FPGAs, effectively reducing production costs.

[0065] like Figure 1 As shown in the figure, an embodiment of the present invention provides a display system for images and videos based on an FPGA-mounted chip, comprising:

[0066] Another object of the present invention is to provide a display system for images and videos based on an FPGA-embedded chip, comprising:

[0067] The host computer is used to periodically send video display commands to the SOPC chip;

[0068] The SOPC chip is used to generate control information for the ADV7341 image encoder.

[0069] The ADV7341 image encoder integrates an I2C slave interface to acquire configuration data from the SOPC chip. It uses the BT656 encoder transmission protocol and works with a video driver to complete display control tasks for different scene images.

[0070] SOPC chips include ARM processors and FPGA display control processors;

[0071] The ARM processor receives video display commands periodically issued by the host computer;

[0072] The ARM processor adopts a block transmission method. For each sub-block of image data transmitted, a wr_done indicator signal and a sub-block image data sequence number are generated for the display control processor FPGA. This triggers the display control processor FPGA to sample the data and store it in the BRAM according to the sequence number, thus storing the pixel control information of a complete sub-block.

[0073] For example, the SOPC chip uses the SM9B100BAL SOPC chip to generate control information for the image encoder ADV7341. The ARM processor inside the SOPC receives image display commands from the host computer software and processes them in a specific format (data packaging, adding synchronization headers) before forwarding the data and commands to the display control processor FPGA via the on-chip FSMC bus. After receiving a complete frame of image information, the display control processor FPGA checks whether the image encoder ADV7341 has been configured. If it has, it triggers the image encoder ADV7341 to work.

[0074] In this embodiment of the invention, the SM9B100BAL SOPC chip is a domestically produced chip with low cost. This method can effectively reduce the number of constraint signals, greatly save the external user IO interface of SOPC, has good scalability, and effectively reduce the PCB area, making it suitable for use in occasions with strict requirements for weight, size and portability.

[0075] In particular, due to the limited register resources of the on-chip compatible XC2V1000 FPGA and the huge amount of image data, this method is not suitable for the strategy of trading area for speed. Instead, it instantiates the internal BRAM resources (using the Xilinx official V6.0 BRAMIP core to instantiate single-port RAM). Due to the flexibility of RAM read and write, the storage resources can be continuously reused in a time-sharing manner.

[0076] Example 3, as Figure 2 As shown, as another embodiment of the present invention, the present invention provides a method for displaying images and videos based on an FPGA-mounted chip, comprising:

[0077] The ARM processor first transmits data to the FPGA, and then transmits commands. The ARM processor uses a block transmission strategy. After each data packet is transmitted, it generates a wr_done indicator signal and a data packet sequence number to the FPGA. This triggers the FPGA to sample the data on the FSMC32-bit internal bus and store it in the BRAM according to the sequence number. The sequence number is not necessarily sequential, because there may be cases where the video background only changes the set area. This sequence number is just used as the starting address for storing in the BRAM.

[0078] Specifically, the video background, for example, is divided into four display areas: A, B, C, and D. Other undefined areas are all white. Each area is further divided into 8 sub-blocks (2 rows and 4 columns), which can adapt to a variety of different display scenarios.

[0079] The display parameters for its background sub-blocks are defined as follows (taking a 32*32 resolution sub-block as an example):

[0080] localparam A_L_X = 11'd417;

[0081] localparam A_R_X = 11'd544;

[0082] localparam A_UP_Y_EVEN=11'd55;

[0083] localparam A_DOWN_Y_EVEN=11'd86;

[0084] localparam A_UP_Y_ODD=11'd367;

[0085] localparam A_DOWN_Y_ODD=11'd398;

[0086] Each region is defined by six parameters: the left boundary, right boundary, upper boundary of even field, lower boundary of even field, upper boundary of odd field, and lower boundary of odd field of the sub-block.

[0087] Specifically, after the display control processor FPGA finishes storing the data of each background sub-block, it sends a wr_ok signal to the ARM processor. The ARM processor detects this signal and then starts the transmission of the next display sub-block. This process is repeated until the number of data exchanged blocks reaches the maximum number of background sub-blocks, MAX_BLOCK_NUM. At this point, the display control processor FPGA generates an all_wr_ok handshake signal to the ARM processor, and the ARM processor starts the transmission of the center cursor motion command. Simultaneously, the display control processor FPGA detects that its own all_wr_ok = 1, starts the encoder state machine, and triggers the image encoder ADV7341.

[0088] like Figure 3 As shown, this embodiment of the invention provides a method for displaying images and videos based on an FPGA-embedded chip. In the encoder BT656 transmission protocol, specifically, the encoder state machine needs to encode the following states:

[0089] (1) IDLE state, encoder idle initialization variable state;

[0090] (2) EAV state (Pixels <= 4): End (not started) synchronization code state, including:

[0091] Output even-field blanking end synchronization code EAV1 = 8'hB6;

[0092] Output even field data end synchronization code EAV2 = 8'h9D;

[0093] Output the odd field blanking end synchronization code EAV4 = 8'hF1;

[0094] Output odd field data end synchronization code EAV5 = 8'hDA;

[0095] (3) VBLANKING state (pixel pointer 5 <= Pixels <= 283): Row blanking state, including:

[0096] Output line blanking area odd pixel position padding code BLANK_ODD = 8'h10;

[0097] Output the padding code for even-numbered pixel positions in the line blanking area: BLANK_EVEN = 8'h80;

[0098] (4) SAV state (pixel pointer 285 <= Pixels <= 288): Start of synchronization code state, including:

[0099] Even field blanking begins synchronization code SAV1;

[0100] Even field data begins synchronization code SAV2;

[0101] Odd field blanking begins synchronization code SAV4;

[0102] Odd Field Data Start Synchronization Code SAV5;

[0103] (5) DATA state (pixel pointer 289 <= Pixels <= 1728): Screen data encoding state, including:

[0104] Luminance component Y1 encoding state: Y_DATA1;

[0105] Chromaticity component Cb encoding status: ODD;

[0106] Luminance component Y2 encoding state: Y_DATA2;

[0107] Chromaticity component Cr encoding status: EVEN;

[0108] (6) END state: The screen has completed the scanning of 1 frame and 2 fields and will jump back to the IDLE state, continuously refreshing the encoded screen.

[0109] Once the FPGA has received a complete frame of image data and the I2C configuration timing for the ADV7341 image encoder has been completed, the START_ENCODING flag signal is pulled high, triggering the image encoder state machine to operate and execute the encoder BT656 transmission protocol. The specific process is as follows:

[0110] Step 1: The encoder state machine is in the IDLE state. Initialize the pixel pointer Pixels = 0 and the line pointer Lines = 0. The encoder data state defaults to the even field blanking end code state EAV. When START_ENCODING = 1, jump to step 2.

[0111] Step 2: The encoder is in WORK state, and the pixel pointer Pixels is incremented by 1 on the falling edge of HCLK. When the EAV state is (Pixels<=4), the preamble codes 8'hff, 8'h00, 8'h00, and EAVx are generated respectively. When the pixel pointer Pixels>4, jump to step 3.

[0112] Step 3: The encoder enters the line blanking area. When the pixel pointer satisfies 5 <= Pixels <= 283, it generates the padding code 8'h10 when the value of the Pixels pointer is odd and generates the padding code 8'h80 when the value of the Pixels pointer is even. When the pixel pointer satisfies Pixels >= 284, it jumps to step 4.

[0113] Step 4: The encoder enters the SAV state and, based on the current number of lines encoded, enters the even field blanking start SAV1, even field data start SAV2, odd field blanking start SAV4, and odd field data start SAV5 states respectively. Similar to step 2, it generates preambles: 8'hff, 8'h00, 8'h00, and SAVx. When the pixel indicator Pixels > 288, it jumps to step 5.

[0114] Step 5: The encoder enters the data encoding DATA state. According to the YCbCr422 encoding format, it sequentially enters the Y_DATA1 state, ODD state, Y_DATA2 state, and EVEN state. When it detects that the pixel pointer Pixels >= 1728, it increments the line index Lines by 1 and sets the pixel pointer Pixels to 1. It then checks whether the current line has reached MAXLINE = 625. If it has not, it jumps to step 2; otherwise, it jumps to step 6.

[0115] Step 6: The encoder has completed the refresh encoding of a complete frame and enters the END state. At this time, the encoded data state datadone is pulled high to indicate that the encoding has been completed. Then, it returns to step 1 and waits for new data or commands, thus achieving the effect of continuously refreshing the screen.

[0116] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0117] The information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0118] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments.

[0119] This invention also provides a computer device comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0120] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps described in the various method embodiments above.

[0121] This invention also provides an information data processing terminal, which, when executed on an electronic device, provides a user input interface to implement the steps described in the above method embodiments. The information data processing terminal is not limited to mobile phones, computers, or switches.

[0122] This invention also provides a server that, when executed on an electronic device, provides a user input interface to implement the steps described in the above method embodiments.

[0123] This invention provides a computer program product that, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments above.

[0124] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0125] To further illustrate the effects of the embodiments of the present invention, the following experiment was conducted: A test device was prepared (supporting the Win7 operating system and CPCI driver, with a host computer interface provided by Nanjing Qinheng Company). This interface can perform single-step data reading and writing at a certain offset address, or perform continuous data reading and writing within a certain area, thereby simulating the real-time transmission of display and control commands from the host computer; a video decoder (Engineering Treasure) was also required to verify the image processor effect of the technical solution described above.

[0126] In the experiment, since the technical solution supports the processing of up to 4 channels of image and video, two engineering tools were used for display verification. The experiment found that the image display commands issued by the host computer can be flexibly and in real time controlled through the technical solution, with smooth picture and no stuttering, meeting the requirements of airborne image display.

[0127] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention and within the spirit and principles of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for displaying images and videos based on an FPGA-embedded chip, characterized in that, This method operates on an SOPC chip, which includes an ARM processor and a display control processor FPGA. The ARM processor receives video display instructions periodically issued by the host computer. The video display instructions include multiple sub-block image data of the video background image after the video background area is planned and divided. The ARM processor adopts a block transmission method. Each time a sub-block of image data is transmitted, the display control processor FPGA generates a wr_done indicator signal and a sub-block image data sequence number, triggering the display control processor FPGA to sample and store the pixel control information of a complete sub-block in the BRAM according to the sequence number. The sequence number is the starting address of the BRAM. After storing the pixel control information of a complete sub-block, the display control processor FPGA replies to the ARM processor with a handshake signal wr_ok and triggers the ARM processor to send the image data of the next sub-block. The handshake operation is continuously repeated until the display control processor FPGA has received and stored the background area information of all videos. Based on the background area information of all received videos, the display control processor FPGA triggers the image encoder state machine to work by configuring the timing of the I2C of the image encoder ADV7341, and completes the display control task of different scene screens through the encoder BT656 transmission protocol. After receiving the video display command, the ARM processor completes the handshake with the display control processor FPGA and the periodic transmission of image data and control commands through the 32-bit FSMC32 bus. The handshake operation includes: the ARM processor sending a wr_done signal to the display control processor FPGA, and the display control processor FPGA receiving the signal and replying with wr_ok, where 1 indicates that it is valid; The video background area planning includes: the display content of a frame image is represented by the pixel black and white control information of the set area. The area planning is represented by sub-blocks in four directions: upper left, upper right, lower left, and lower right. The pixel black and white control includes: 1 from the SOPC chip indicates that a specific pixel is lit, and 0 indicates that a specific pixel is not lit. The display control processor FPGA decodes 1 as: 8'h10 black lit, and decodes 0 as: 8'hf0 white default state.

2. The method for displaying images and videos based on an FPGA-equipped chip according to claim 1, characterized in that, The ARM processor achieves periodic transmission of the 32-bit FSMC32 bus by configuring frequency division, frequency multiplication, and selecting the enable register.

3. The method for displaying images and videos based on an FPGA-mounted chip according to claim 1, characterized in that, After receiving the background area information of all videos, the display control processor FPGA checks whether the image encoder ADV7341 has been configured. If it has, the image encoder ADV7341 is triggered to work.

4. The method for displaying images and videos based on an FPGA-mounted chip according to claim 1, characterized in that, The display control processor FPGA completes the register configuration of the internal working mode of the image encoder ADV7341 by simulating I2C communication timing. The specific registers configured are: chip reset mode control register, chip power supply mode control register, SD standard definition control register, YCbCr mode register, single 8-bit encoder output control register, internal color bar test disable register, output selection register, subcarrier frequency control register, and PAL / NTSC format selection register. The internal operating mode of the image encoder ADV7341 includes the generation of the encoder operating clock. The encoder operating clock is the operating clock for the image encoder ADV7341 to perform internal DAC conversion by image digital encoding. It is a 27MHz clock, which includes the CLOCK, HCLK and CLKOUT clock signals.

5. The method for displaying images and videos based on an FPGA-mounted chip according to claim 4, characterized in that, The image encoder ADV7341 uses synchronization code EAV, with SAV serving as the synchronization information for encoder encoding.

6. The method for displaying images and videos based on an FPGA-embedded chip according to claim 1, characterized in that, The specific process of the encoder BT656 transmission protocol is as follows: Step 1: The encoder state machine is in the IDLE state. Initialize the pixel pointer Pixels=0 and the line pointer Lines=0. The encoder data state defaults to the even field blanking end code state EAV. When START_ENCODING=1, jump to step 2. Step 2: The encoder is in WORK state, and the pixel pointer Pixels is incremented by 1 on the falling edge of HCLK. In EAV state, when Pixels<=4, the preamble is generated respectively: 8'hff, 8'h00, 8'h00, and EAVx. When the pixel pointer Pixels>4, jump to step 3. Step 3: The encoder enters the line blanking area. When the pixel pointer satisfies 5 <= Pixels <= 283, it generates the padding code 8'h10 when the value of the Pixels pointer is odd and generates the padding code 8'h80 when the value of the Pixels pointer is even. When the pixel pointer satisfies Pixels >= 284, it jumps to step 4. Step 4: The encoder enters the SAV state and, based on the current number of lines encoded, enters the even field blanking start SAV1, even field data start SAV2, odd field blanking start SAV4, and odd field data start SAV5 states respectively. Similar to step 2, it generates preambles: 8'hff, 8'h00, 8'h00, and SAVx. When the pixel indicator Pixels > 288, it jumps to step 5. Step 5: The encoder enters the data encoding DATA state. According to the YCbCr422 encoding format, it sequentially enters the Y_DATA1 state, ODD state, Y_DATA2 state, and EVEN state. When it detects that the pixel pointer Pixels>=1728, it increments the line index Lines by 1 and sets the pixel pointer Pixels to 1. It then checks whether the current line has reached MAXLINE=625. If it has not, it jumps to step 2; otherwise, it jumps to step 6. Step 6: The encoder has completed the refresh encoding of a complete frame and enters the END state. At this time, the encoded data status datadone is pulled high to indicate that the encoding has been completed. Then, it returns to Step 1 and waits for new data or commands to continuously refresh the screen.

7. A display system for images and videos based on an FPGA-embedded chip, characterized in that, The system is implemented using the image and video display method based on an FPGA-mounted chip as described in any one of claims 1-6. The system includes: The host computer is used to periodically send video display commands to the SOPC chip; The SOPC chip is used to generate control information for the ADV7341 image encoder. The ADV7341 image encoder integrates an I2C slave interface to acquire configuration data from the SOPC chip. It uses the BT656 encoder transmission protocol and works with a video driver to complete display control tasks for different scene images.

8. The image and video display system based on an FPGA-embedded chip according to claim 7, characterized in that, SOPC chips include ARM processors and FPGA display control processors; The ARM processor receives video display commands periodically issued by the host computer; The ARM processor adopts a block transmission method. For each sub-block of image data transmitted, a wr_done indicator signal and a sub-block image data sequence number are generated for the display control processor FPGA. This triggers the display control processor FPGA to sample the data and store it in the BRAM according to the sequence number, thus storing the pixel control information of a complete sub-block.

Images