A nationalization-based image decoding unit and a working method thereof

Abstract

The application discloses a nationalization-based image decoding unit and a working method thereof, wherein the image decoding unit comprises a processing unit, an interface unit and a power supply system; the processing unit comprises a domestic RK3588 processor and an FPGA chip; the interface unit comprises connectors XS1-XS5; and the power supply system is used for supplying power for the RK3588 processor and the FPGA chip. The application further discloses a working method of the nationalization-based image decoding unit. In one aspect, the combination of the domestic RK3588 processor and the FPGA chip can realize multi-channel video decoding; the PAL interface for external video output can be specially designed for technical compatibility of analog and digital displays; in another aspect, data flow and image output are optimized, processing delay is reduced, the low-delay system requirement is ensured, and more video processing can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image decoding, and in particular to a nationally produced image decoding unit and a working method thereof. Background Art

[0002] Video transmission systems require not only stable real-time data transmission but also low latency, high image quality, and flexible decoding capabilities. To meet these requirements, current solutions on the market generally use high-speed video interface output. However, in certain specific application scenarios, existing technical solutions cannot meet specific needs.

[0003] Furthermore, existing decoding solutions commonly suffer from several technical drawbacks. First, many existing video decoding output interfaces only support common digital video interfaces, not traditional analog interfaces like PAL (Phase Alternating Line). This makes existing technologies inapplicable in scenarios requiring traditional video output. Second, existing solutions suffer from significant transmission latency, making them inadequate for some applications requiring high real-time performance. This is particularly true for applications such as in-vehicle and airborne systems, where real-time video quality is crucial. Excessive latency can affect system responsiveness and even security, significantly limiting the technology's application. Furthermore, while existing video processing solutions based solely on FPGAs (field programmable gate arrays) offer a certain degree of customizability, they struggle to support a wide range of video processing tasks, such as video watermarking, outputting actual image resolution information, and extracting the Y-byte component. This makes existing technologies insufficient for demanding applications and unable to meet the growing market demand. Summary of the Invention

[0004] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide a domestically produced image decoding unit and its working method that is compatible with traditional analog interfaces, has low latency, high real-time performance and powerful video processing capabilities.

[0005] The technical solution of the present invention is:

[0006] The present invention is based on a domestically produced image decoding unit, comprising a processing unit, an interface unit and a power supply system; the processing unit comprises a domestically produced RK3588 processor and an FPGA chip;

[0007] The FPGA chip includes:

[0008] SPI input interface, connected to external devices through the interface unit, used to receive code stream data sent by external devices;

[0009] SPI output interface, connected to the RK3588 processor, used to output code stream data to the RK3588 processor;

[0010] Image input interface, connected to the RK3588 processor, used to receive image data decoded by the RK3588 processor;

[0011] Read and write interface, used to read and write image data decoded by RK3588 processor;

[0012] PAL output interface, connected to an external device through an interface unit, for sending stored image data to the external device;

[0013] PAL input interface, used to receive PAL input signal and provide synchronous clock and control signal;

[0014] The output end of the RK3588 processor is connected to the FPGA chip through a multi-channel PCIE interface, and is used to send the decoded image data to the FPGA chip.

[0015] Furthermore, the FPGA chip further includes:

[0016] Synchronous output serial port: connected to external devices through the interface unit, used to forward the image data received after decoding by the RK3588 processor;

[0017] GPIO interface, connected to the RK3588 processor, used for interrupting and resetting the RK3588 when reading the code stream data in the FIFO buffer via the SPI interface.

[0018] Furthermore, the interface unit includes a connector XS1, which is the internal data port of the image decoding unit and is used to power the image decoding unit; the connector XS1 is connected to the power supply system; the output end of the external device is connected to at least two SPI input interfaces of the FPGA chip via the connector XS1; the connector XS1 is also connected to one UART interface of the FPGA chip for debugging; the FPGA chip also outputs to the RS422 conversion chip via at least two synchronous serial ports, and the RS422 level is synchronized through the RS422 conversion chip and connected to the external device via the connector XS1 to forward the received image data decoded by the RK3588 processor.

[0019] Furthermore, the interface unit includes connectors XS2 to XS4; the connectors XS2 and XS3 are analog video output ports, and the connector XS4 is a synchronization signal input port; the output end of the RK3588 processor is connected to the FPGA chip via a BT656 interface; the FPGA chip is connected to the connector XS2 via a first conversion chip, and is also connected to the connector XS3 via a second conversion chip. The first conversion chip and the second conversion chip are used to convert the RGB signal format into a PAL signal and send it to an external device; the connector XS4 is connected to the input end of the FPGA chip via a third conversion chip. The third conversion chip is used to convert the external PAL signal into a BT656 signal format and send it to the FPGA chip.

[0020] Furthermore, the interface unit includes a connector XS5, which is the program upgrade port of the image decoding unit and is used for program debugging and upgrading. The RK3588 processor and FPGA chip are electrically connected to the connector XS5 through various debugging interfaces; a Gigabit Ethernet is also connected between the RGMII port of the RK3588 processor and the connector XS5, which is used as a debugging network port.

[0021] A working method based on a nationally produced image decoding unit of the present invention comprises the following steps:

[0022] S1: FPGA receives the bit stream data transmitted by the external device through at least one communication interface, stores it in the internal buffer after processing, and sends the bit stream data to be processed in the buffer to the RK3588 processor after receiving the instruction;

[0023] S2: The RK3588 processor receives the code stream data, decodes the data, and sends the decoded image data back to the FPGA for storage;

[0024] S3: FPGA determines whether it has received an external PAL input signal. If it has, FPGA reads and outputs the stored image data based on the PAL input signal. If it has not, FPGA simulates the PAL input signal timing, reads and outputs the stored image data.

[0025] Furthermore, in S1, the process of transmitting and receiving code stream data by the FPGA includes:

[0026] S1-1: FPGA receives serial code stream data transmitted by an external device through at least one serial communication interface;

[0027] S1-2: The FPGA samples the received serial code stream data in real time using the clock signal of the serial communication interface, converts the sampled serial code stream data into parallel code stream data, and stores the data into an internal buffer;

[0028] S1-3: After receiving the control command sent by the RK3588 processor, the FPGA returns the valid length information of the code stream data to be processed in the buffer area to the RK3588 processor;

[0029] S1-4: According to the received data length, the RK3588 processor reads the code stream data of the corresponding length by controlling the clock signal, and decodes the received code stream data into original image data, thereby completing the processing of the data.

[0030] Further, in S1 and S2, the FPGA receives at least two SPI serial data code streams from an external device, stores the processed data in the FIFO buffer area, and after receiving the control command sent by the RK3588 processor, forwards the code stream data in the FIFO buffer area to the RK3588 processor through the SPI interface; after receiving the code stream data, the RK3588 processor performs data decoding processing, and transmits the decoded image data back to the FPGA in the form of H2C through the PCIE interface; after receiving the image data transmitted by the RK3588 processor, the FPGA stores the image data in the internal memory.

[0031] Further, in S2, the process of the FPGA receiving image data includes: before performing the read and write operations of the memory, the FPGA buffers the previous stage data through the FIFO buffer area; then, the read and write operations of the memory are performed in a ping-pong operation mode, and when writing data, data reading is performed in a bank different from the current writing operation to avoid read-write conflict.

[0032] Further, in S3, the process specifically includes the following steps: the FPGA judges whether the external PAL input signal is received; if the PAL input signal is received, the image data is read from the memory based on the timing of the PAL input signal, and the image data is output through at least two PAL output signals; if the PAL input signal is not received, the FPGA simulates the timing of the PAL input signal, reads the image data from the memory according to the simulated PAL timing, and outputs the image data through at least two PAL output signals.

[0033] The beneficial effects of the present application are as follows:

[0034] (1) The combination of the domestic RK3588 processor and the FPGA chip can perform multi-channel video decoding;

[0035] (2) The PAL interface for external video output can be specially designed for technical compatibility with analog and digital displays;

[0036] (3) Using FPGA as clock intermediary, direct data transmission of PCIE interface, application of ping-pong operation on DDR3 and PAL signal synchronization technology, together optimize data flow and image output, reduce processing delay and ensure low latency system requirements;

[0037] (4) By setting up multiple interfaces, more video processing can be achieved; for example: video watermark overlay, actual image resolution information output, Y byte component extraction, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 1 is a schematic diagram of the circuit principle of the image decoding unit according to an embodiment of the present invention;

[0039] Figure 2 This is a simplified schematic diagram of the working method of the image decoding unit according to an embodiment of the present invention;

[0040] Figure 3 This is a block diagram of the structure of the FPGA chip according to an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of the code stream receiving and sending process of the FPGA chip according to an embodiment of the present invention;

[0042] Figure 5 FIG1 is a flow chart of an FPGA receiving image data according to an embodiment of the present invention;

[0043] Figure 6 1 is a schematic diagram of a process for receiving image data by PAL according to an embodiment of the present invention;

[0044] Figure 7 1 is a schematic diagram of a specific PAL output timing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] like Figure 1 Figure 1 shows a domestically produced image decoding unit, comprising a processing unit, an interface unit, and a power supply system. The processing unit comprises a domestically produced RK3588 processor and an FPGA chip. The interface unit comprises connectors XS1 to XS5. The power supply system is used to supply power to the RK3588 processor and the FPGA chip.

[0047] In this embodiment, the RK3588 processor is a high-end, general-purpose SOC manufactured using an 8nm process. Its CPU utilizes an octa-core architecture consisting of four A76 and four A55 cores. Its GPU (Image Processing Unit) utilizes the Mail G610 MP4, integrated with a 6TOPS independent NPU, and supports four channels of LPDDR4 / 5, effectively guaranteeing the RK3588's high-performance output. The RK3588 also excels in video encoding and decoding, supporting hardware encoding and decoding of 8K video, up to 8K@60fps decoding and output, providing stable, high-definition images for VR applications. The RK3588 utilizes an independent hardware decoder, resulting in higher decoding efficiency and significantly lower power consumption. This reduces CPU resource consumption, freeing the CPU to process other applications and ensuring high performance. Furthermore, the RK3588 supports the latest LPDDR5 memory chips and utilizes a four-channel RAM design, fully guaranteeing the full performance of the RK3588 platform. The RK3588's inherent computing power can be leveraged for post-processing of camera-captured data, enabling image optimization, splicing, and overlay processing.

[0048] In this embodiment, according to the technical protocol requirements of the image decoding unit, both the synchronous input interface and the decompressed video output interface are PAL signals. However, the video output interface of the RK3588 cannot support the requirements of the current technical protocol. For these reasons, this embodiment uses an FPGA chip to complete the video data format conversion and interface design. After comprehensively evaluating the logic resources required to implement the interface requirements of the technical protocol and the requirements for localization, Fudan Micro's JFM7K325T-N was selected as the FPGA chip for the video display processing board.

[0049] The main interface requirements of the FPGA chip in this embodiment are as follows:

[0050] 1) Two SPI input interfaces: used to receive code stream data sent by external devices and store it in the FIFO of the FPGA chip;

[0051] 2) Two SPI output interfaces: used to enable the RK3588 processor to read the code stream data stored in the FIFO;

[0052] 3) Two synchronous serial port outputs: used to forward the image data received after decoding by the RK3588 processor;

[0053] 4) Two MIPI input interfaces: used to receive image data decoded by the RK3588 processor;

[0054] 5) Standard DDR3 read / write interface: used to read and write image data decoded by the RK3588 processor;

[0055] 6) Two PAL output interfaces: used to send image data stored in DDR3 to external devices;

[0056] 7) One PAL input interface: used to provide clock and control signals for PAL output timing;

[0057] 8) Four GPIO interfaces: used for interruption and reset of two RK3588s reading FIFO stream data via SPI.

[0058] In this embodiment, the specific connection structure between the RK3588 processor and the FPGA chip is as follows: the FPGA chip's two SPI output interfaces are connected to the RK3588 processor's input terminals, allowing the RK3588 processor to read the bitstream data stored in the FIFO; the FPGA chip's four GPIO interfaces are connected to the RK3588 processor; the FPGA chip and the RK3588 processor communicate via the I2C bus; the FPGA chip's UART interface is connected to the RK3588 processor's UART interface for serial communication; the RK3588 processor is connected to the FPGA chip via four PCIE interfaces, utilizing PCIE's Host-to-Card (H2C) method to logically divide the decoded image data into two paths and transmit them in parallel to the FPGA for processing. The RK3588 processor's output terminal is connected to the FPGA chip via a BT656 interface for transmitting video data for subsequent processing. The RK3588 processor's MIPI interface is connected to the FPGA chip for sending the decoded image data to the FPGA chip.

[0059] The above connection structure embodies the close cooperation between the FPGA chip and the RK3588 processor through multiple communication interfaces, aiming to provide efficient data transmission and processing capability. Each interface plays a unique role and cooperates with each other to ensure the high performance and stability of the system. Specifically, the SPI interface provides high-speed serial communication, which can efficiently transmit code stream data from the FIFO of the FPGA to the RK3588 processor. The use of two-way SPI interface increases the bandwidth of data transmission, ensuring fast and stable reading of data. The GPIO interface provides a flexible signal control method, which can be used for triggering, state monitoring and transmission of control signals. Through four-way GPIO, low-delay control and state exchange can be performed between the FPGA and the RK3588 processor; through the I2C bus, the FPGA and the RK3588 processor can realize functions such as parameter configuration and device initialization; UART is used for serial data communication, which is suitable for low-speed and reliable serial transmission. In this design, the UART connection between the FPGA and the RK3588 can be used for system-level debugging, log transmission or low-speed data exchange; PCIE provides high-bandwidth and high-throughput data transmission capability, especially suitable for applications that require large amounts of parallel processing. Through the PCIE interface, the RK3588 can transmit the decoded image data to the FPGA in two parallel ways in the Host-to-Card mode, thereby greatly improving the processing efficiency and data throughput of the system. BT656 is a standard video transmission protocol widely used in video decoding and display devices. Through the BT656 interface, the RK3588 can efficiently transmit decoded image data to the FPGA, which is suitable for video data processing application scenarios. MIPI is a high-speed serial interface, through which the RK3588 can transmit image data to the FPGA.

[0060] In this embodiment, the RK3588 processor can also be directly connected to the HDMI device through the HDMI interface via the HDMI connector.

[0061] In this embodiment, the connector XS1 is an internal data port of the image decoding unit, and is used to power the image decoding unit. Specifically, the power supply system powers the image decoding unit through the connector XS1, adopts +5V DC input, and generates voltages required by each device through a DC / DC conversion chip or an LDO chip. In addition, the output end of the external device is connected to two-way SPI input interfaces of the FPGA chip through the connector XS1, which is used to send code stream data to the FPGA chip; the connector XS1 is connected to a one-way UART interface of the FPGA chip, which is used for debugging; the FPGA chip is also output to the conversion chip through two-way synchronous serial ports, and the conversion chip is used to realize the connection of the synchronous RS422 level with the external device through the connector XS1, so as to forward the received image data decoded by the RK3588 processor, and the conversion chip is preferably the HR3490 chip of Beijing Huaxin Micro.

[0062] In this embodiment, the connectors XS2 and XS3 are analog video output ports, and the connector XS4 is a synchronous signal input port. Among them, the synchronous signal input port and the analog video output port are both PAL signals, the FPGA chip is connected to the connector XS2 through a first conversion chip, so as to facilitate the communication of the corresponding external device. The FPGA chip is connected to the connector XS3 through a second conversion chip; when the FPGA chip receives the image data of the RK3588 processor, it is respectively connected to each conversion chip, so as to be converted into the analog signal output required by the external device, and the corresponding external device can be a display. Among them, the first conversion chip and the second conversion chip both adopt the GM7392 chip, which is used to convert the RGB signal format into the PAL signal and send it to the display. The connector XS4 is connected to the input end of the FPGA chip through a third conversion chip, and the third conversion chip is used to convert the external PAL signal into the BT656 signal format and send it to the FPGA chip; the third conversion chip selects the GM7150A of Chengdu Zhenxin.

[0063] In this embodiment, connector XS5 is the program upgrade port of the image decoding unit, which is used for program debugging and upgrading. Specifically, the RK3588 processor and the FPGA chip are electrically connected to the connector XS5 through various debugging interfaces. For example: a Gigabit Ethernet is connected between the RGMII port of the RK3588 processor and the connector XS5, which is used as a debugging network port, and can realize external communication through a 100Mbps Ethernet interface. The RK3588 processor is connected to the connector XS5 line through a dedicated USB download interface, and can upgrade the system or UBOOT. An external PC can download the file system or IMG file required by the RK3588 processor, which is convenient for customers to upgrade. The RK3588 processor prints the system startup log through a dedicated UART debugging interface, and an external PC can observe the startup process of some RK3588 systems, which is convenient for customers to debug. For another example: the FPGA chip is electrically connected to the connector XS5 through the JTAG debugging port to provide debugging access.

[0064] In this embodiment, both the RK3588 processor and the FPGA chip are equipped with memory chips. The RK3588 processor provides two 32-bit LPDDR4x memory controller interfaces. Considering the board size and technical requirements for capacity, a 32-bit LPDDR4x is selected. Each channel requires only one 32-bit memory chip, so a total of two memory chips are used, forming two 32-bit dual-channel 8GB memory capacities. The memory controller bandwidth can reach 29.8GB / s. This memory chip uses a board-mounted design instead of the traditional memory slot plus memory stick approach to enhance the board's resistance to vibration and shock. The FPGA chip is equipped with 1GB of DDR3 memory for video data caching. The preferred model is two XZF41J256M16 chips from Zhongtianxing Control.

[0065] Both the RK3588 processor and FPGA chip are equipped with memory chips. The RK3588 storage chip features 32GB of EMMC memory for storing the file system, operating system, and upgradeable programs. The FPGA chip is configured with NOR Flash, specifically one EFM25F128A chip with a 16MB capacity, for storing FPGA configuration files.

[0066] like Figure 2 and Figure 3As shown, the working method of this embodiment is as follows: First, the FPGA receives two SPI code streams and forwards them to the RK3588 processor via the SPI interface. The RK3588 processor decodes the received code streams and transmits the image data to the FPGA chip via the PCIE interface in two H2C channels. After receiving the image data, the FPGA stores it in the DDR3 memory. Next, the FPGA determines whether there is an external PAL_IN signal input. If there is a PAL_IN signal input, the FPGA will read the stored image data from the DDR3 memory according to the timing of the PAL_IN signal and output it through the two PAL_OUT interfaces. If there is no PAL_IN signal input, the FPGA simulates the PAL_IN timing and outputs the image data stored in the DDR3 memory through the two PAL_OUT interfaces.

[0067] The above technology ensures the system's compatibility. When the system receives an external PAL_IN signal, the FPGA synchronizes image output according to the signal's timing. This ensures that image data is displayed according to standard PAL timing, avoiding image delays or distortion caused by timing asynchrony. Furthermore, when no external PAL_IN signal is present, the FPGA simulates the PAL_IN signal's timing to output images, further enhancing system compatibility. This means that even without an external synchronization signal, the system can still output image data that complies with the PAL standard, ensuring stable operation in a variety of environments and enhancing the system's versatility and compatibility.

[0068] like Figure 4 As shown in the figure, in this embodiment, the FPGA code stream transmission and reception process is as follows: the FPGA chip first receives the serial code stream data transmitted by the SPI interface, then samples the data using the rising edge of the SPI clock, converts the sampled serial code stream data into parallel code stream data, and stores it in the FIFO buffer. Next, after receiving the control command of the RK3588 processor, the FPGA returns the length of the data to be processed in the FIFO to the RK3588. After obtaining the data length, the RK3588 controls the clock signal to read the code stream data of the corresponding length and decodes the received code stream data into the original image, completing the data processing.

[0069] The reason why we choose to use FPGA as an intermediary instead of directly transmitting SPI data to the RK3588 processor is that the bit rate of the external SPI input is not fixed, and the SPI clock rate may fluctuate, causing the RK3588 to be unable to effectively process the variable-rate SPI signal. FPGA as an intermediary can solve this problem of unstable clock rate and ensure the stability of data transmission and decoding.

[0070] like Figure 5As shown in the figure, in this embodiment, the image data receiving process of the FPGA is as follows: after receiving the PCIE data, the FPGA writes the data into the DDR3 memory. Before reading and writing the DDR3, the previous stage avoids simultaneous reading and writing operation through the FIFO buffer to ensure the consistency of the data. In order to ensure the integrity of the image and solve the image tearing problem, the FPGA adopts the ping-pong operation to read and write the DDR3. In this way, when writing data (for example, writing into Bank 1), the data can only be read from another Bank (for example, Bank 2), so as to avoid the read-write conflict.

[0071] The RK3588 and the FPGA perform data transmission through the PCIE, and the PCIE can directly access the memory of the FPGA. Therefore, the decoded data does not need to be additionally buffered and does not need to be processed across the clock domain, and can be directly sent to the FPGA. Since the frame rate of the PAL signal is 50 frames per second, and the interlaced scanning mode is adopted, it is equivalent to that 40 ms of time is needed for transmitting one frame of picture, so that the transmission delay can be effectively reduced, and the demand of low-delay application can be met.

[0072] As shown in the figure, in this embodiment, the image data receiving process of the FPGA is as follows: after receiving the PCIE data, the FPGA writes the data into the DDR3 memory. Before reading and writing the DDR3, the previous stage avoids simultaneous reading and writing operation through the FIFO buffer to ensure the consistency of the data. In order to ensure the integrity of the image and solve the image tearing problem, the FPGA adopts the ping-pong operation to read and write the DDR3. In this way, when writing data (for example, writing into Bank 1), the data can only be read from another Bank (for example, Bank 2), so as to avoid the read-write conflict. Figure 6

[0073] The synchronization function of the PAL_IN signal is mainly used for image superposition. When the output PAL picture needs to be superimposed with an image, other external images can be conveniently superimposed on the current picture without additional complex algorithm processing, which is helpful to simplify the system design and improve the stability of the system. In the case where external synchronization is not needed, the PAL_IN signal can not be connected. When the FPGA detects that there is no PAL_IN input, the image data is output through the simulation of the PAL_IN timing.

[0074] The specific PAL output timing is shown in the figure. The PAL output adopts the standard BT656 data format, the resolution is 720*576, the frame rate is 50 bps, and the scanning mode is interlaced scanning. Figure 7

[0075] In summary, the application has the following advantages:

[0076] ​​(1) High-efficiency video decoding and processing capability: using domestic RK3588 processor for two-way video decoding means that the system has the ability of parallel decoding, which enables the system to process video streams from different sources simultaneously and ensures efficient data decoding; RK3588 processor usually has strong multi-core processing capability, which is suitable for processing high-concurrency video data stream and improves the overall performance of the system; in addition to decoding function, RK3588 processor can also realize more video processing tasks combined with FPGA, such as video watermark superposition, resolution conversion, color adjustment, etc., which can be applied to decoded video stream in real time to provide higher quality output;

[0077] (2) PAL interface output and compatibility design The design of PAL interface enables the system to simultaneously support traditional analog display and digital display. Through the compatibility of hardware design, it can automatically adapt to the input requirements of different display devices without additional device conversion. This technical compatibility design greatly simplifies the interface design of the system and expands the application range of the device. ② Signal format conversion: through the automatic conversion of each conversion chip, the video output is converted into analog signals conforming to the PAL standard, and high-quality digital video output is also provided, ensuring that the display device can correctly display the decoded video stream;

[0078] (3) FPGA as a clock intermediary, direct data transmission of the PCIE interface, application of ping-pong operation on DDR3, and PAL signal synchronization technology jointly optimize data flow and image output, reduce processing delay, and ensure low-latency system requirements. Specifically: ① FPGA as a clock intermediary solves the problem of SPI clock instability: After receiving the SPI code stream, the FPGA uses its internal stable clock to sample and convert the data, and forwards it to the RK3588 processor with stable timing. In this way, the FPGA effectively solves the timing problem caused by SPI clock fluctuations, thereby ensuring the stability of the data flow and reducing the delay in transmission. ② Data transmission optimization of the PCIE interface: The RK3588 processor transmits decoded image data between the PCIE interface and the FPGA. PCIE has high-speed transmission capabilities and can directly access the FPGA's memory, avoiding additional caching or data transmission across clock domains. Therefore, image data can be directly transferred from the RK3588 to the FPGA for processing without going through complex buffering or additional synchronization mechanisms, reducing intermediate steps and unnecessary delays. ③ Using Ping-Pong Operations to Avoid DDR3 Read / Write Conflicts: To ensure data consistency and prevent data tearing, the FPGA uses a ping-pong operation for DDR3 reads and writes. In this mode, while data is being written to one DDR3 memory bank, data is being read from another bank. This method effectively avoids read / write conflicts, ensuring image data integrity. Parallel read and write operations also improve processing speed and reduce latency caused by waiting and synchronization. ④ PAL Signal Frame Rate and Interlaced Scanning Technology: When outputting PAL signals, the image frame rate is 50 frames per second, with each frame transmitted in 40ms, effectively reducing transmission latency. Interlaced scanning technology allows each frame to be transmitted in just 40ms, enabling faster image data output compared to progressive scanning, thus meeting low-latency application requirements. ⑤ FPGA Synchronizes Image Output Based on the PAL_IN Signal: Upon detecting the external PAL_IN signal, the FPGA synchronizes image output according to the PAL_IN signal timing, thus avoiding latency and image tearing caused by image desynchronization. When there is no PAL_IN signal, the FPGA will output the image by simulating the timing of the PAL_IN signal, further optimizing the delay performance and ensuring that the system can output image data with stable timing even in the absence of an external synchronization signal.

[0079] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A nationally produced image decoding unit comprising a processing unit, an interface unit, and a power supply system; characterized in that: The processing unit includes a domestically produced RK3588 processor and an FPGA chip; The FPGA chip includes: SPI input interface, connected to external devices through the interface unit, used to receive code stream data sent by external devices; SPI output interface, connected to the RK3588 processor, used to output code stream data to the RK3588 processor; Image input interface, connected to the RK3588 processor, used to receive image data decoded by the RK3588 processor; Read and write interface, used to read and write image data decoded by RK3588 processor; PAL output interface, connected to an external device through an interface unit, for sending stored image data to the external device; PAL input interface, used to receive PAL input signal and provide synchronous clock and control signal; The output end of the RK3588 processor is connected to the FPGA chip through a multi-channel PCIE interface, and is used to send the decoded image data to the FPGA chip; The interface unit includes connectors XS1 to XS5; The connector XS1 is the internal data port of the image decoding unit and is used to power the image decoding unit; the connector XS1 is connected to the power supply system; the output end of the external device is connected to at least two SPI input interfaces of the FPGA chip via the connector XS1; the connector XS1 is also connected to one UART interface of the FPGA chip for debugging; the FPGA chip also outputs to the RS422 conversion chip via at least two synchronous serial ports, and the RS422 level is synchronized through the RS422 conversion chip to connect to the external device via the connector XS1 to forward the received image data decoded by the RK3588 processor; Connectors XS2 and XS3 are analog video output ports, and connector XS4 is a synchronization signal input port. The output of the RK3588 processor is connected to the FPGA chip via a BT656 interface. The FPGA chip is connected to connector XS2 via a first conversion chip and to connector XS3 via a second conversion chip. The first and second conversion chips are used to convert RGB signal formats into PAL signals and send them to external devices. Connector XS4 is connected to the input of the FPGA chip via a third conversion chip. The third conversion chip is used to convert external PAL signals into BT656 signal format and send them to the FPGA chip. The connector XS5 is the program upgrade port of the image decoding unit, which is used for program debugging and upgrading. The RK3588 processor and FPGA chip are electrically connected to the connector XS5 through various debugging interfaces. A Gigabit Ethernet is also connected between the RGMII port of the RK3588 processor and the connector XS5, which is used as a debugging network port.

2. The nationalized image decoding unit according to claim 1, characterized in that: The FPGA chip further includes: Synchronous output serial port: connected to external devices through the interface unit, used to forward the image data received after decoding by the RK3588 processor; GPIO interface, connected to the RK3588 processor, used for interrupting and resetting the RK3588 when reading the code stream data in the FIFO buffer via the SPI interface.

3. A working method based on a fully domesticated image decoding unit according to claim 1, characterized in that: The following steps are involved: S1: FPGA receives the bit stream data transmitted by the external device through at least one communication interface, stores it in the internal buffer after processing, and sends the bit stream data to be processed in the buffer to the RK3588 processor after receiving the instruction; S2: The RK3588 processor receives the code stream data, decodes the data, and sends the decoded image data back to the FPGA for storage; S3: FPGA determines whether it has received an external PAL input signal. If it has, FPGA reads and outputs the stored image data based on the PAL input signal. If it has not, FPGA simulates the PAL input signal timing, reads and outputs the stored image data.

4. The working method based on the nationalized image decoding unit according to claim 3, characterized in that: In S1, the process of transmitting and receiving code stream data of the FPGA includes: S1-1: FPGA receives serial code stream data transmitted by an external device through at least one serial communication interface; S1-2: The FPGA samples the received serial code stream data in real time using the clock signal of the serial communication interface, converts the sampled serial code stream data into parallel code stream data, and stores the data into an internal buffer; S1-3: After receiving the control command sent by the RK3588 processor, the FPGA returns the valid length information of the code stream data to be processed in the buffer area to the RK3588 processor; S1-4: The RK3588 processor reads the code stream data of corresponding length by controlling the clock signal according to the length of the received data, and decodes the received code stream data into original image data to complete the data processing.

5. The working method based on the nationalized image decoding unit according to claim 3 or 4, characterized in that: In S1 and S2, the FPGA receives at least two SPI serial data streams from external devices, stores them in the FIFO buffer after processing, and forwards the stream data in the FIFO buffer to the RK3588 processor through the SPI interface after receiving the control command sent by the RK3588 processor. After receiving the stream data, the RK3588 processor decodes the data and transmits the decoded image data back to the FPGA through the PCIE interface in H2C mode. After receiving the image data transmitted by the RK3588 processor, the FPGA stores it in its internal memory.

6. The working method based on the nationalized image decoding unit according to claim 5, characterized in that: In S2, the process of FPGA receiving image data includes: before performing memory read and write operations, FPGA caches the previous level of data through the FIFO buffer; then uses the ping-pong operation to read and write the memory. When writing data, it selects a different bank from the current write operation to read data to avoid read and write conflicts.

7. The working method based on the nationalized image decoding unit according to claim 5, characterized in that: S3 specifically includes the following steps: the FPGA determines whether an external PAL input signal is received; if a PAL input signal is received, the FPGA reads image data from the memory based on the timing of the PAL input signal and outputs the image data through at least two PAL output signals; if no PAL input signal is received, the FPGA simulates the timing of the PAL input signal, reads image data from the memory according to the simulated PAL timing, and outputs the image data through at least two PAL output signals.

Images