Full-localization-based image decoding unit and working method thereof

Abstract

The invention discloses a full-localization-based image decoding unit and a working method thereof. The image decoding unit comprises a processing unit, an interface unit and a power supply system, the processing unit comprises a domestic RK3588 processor and an FPGA (Field Programmable Gate Array) chip; the interface unit comprises a connector XS1, a connector XS2 and a connector XS5; and the power supply system is used for supplying power to the RK3588 processor and the FPGA chip. The invention further comprises a working method based on the full-localization image decoding unit. According to the invention, on one hand, multi-channel video decoding can be carried out by using the combination of the localized RK3588 processor and the FPGA chip; an external video output PAL interface can be subjected to special technology compatible design aiming at analog and digital displays; and on the other hand, data flow and image output are optimized, the processing delay is reduced, the low-delay system requirement is ensured, and more video processing can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image decoding, and in particular to a nationally localized image decoding unit and a working method thereof. Background Art

[0002] The video transmission system requires not only stable real-time data transmission, but also low latency, high image quality and flexible decoding capabilities. In order to meet these requirements, the current solutions on the market generally use high-speed video interface output. However, in some specific application scenarios, the existing technical solutions cannot meet specific needs.

[0004] In addition, existing decoding solutions generally have some technical defects. First, many existing video decoding output interfaces only support common digital video interfaces, but not traditional analog interfaces such as PAL (Phase Alternating Line), which makes existing technologies inapplicable in some scenarios that require traditional video output. Secondly, the existing solutions have large transmission delays and cannot meet some application scenarios with high real-time requirements, especially in vehicle-mounted and airborne applications where video real-time requirements are strict. The delay problem is particularly prominent. Excessive delay will affect the response speed of the system and even affect security, becoming an important factor restricting the application of technology. Furthermore, although the existing video processing solutions based solely on FPGA (field programmable gate array) have certain customizability, they are difficult to support more video processing tasks, such as video watermark superposition, actual image resolution information output, Y byte component extraction, etc. This makes the existing technology appear to be powerless in some high-demand applications and unable to meet the growing market demand. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned deficiencies in the prior art and provide a nationally produced image decoding unit and a working method thereof that is compatible with traditional analog interfaces, has low latency, high real-time performance and powerful video processing capabilities.

[0006] The technical solution of the present invention is: The present invention is based on a domestically produced image decoding unit, comprising a processing unit, an interface unit and a power supply system; the processing unit comprises a domestically produced RK3588 processor and an FPGA chip; The FPGA chip comprises: SPI input interface, connected to an external device through an interface unit, for receiving code stream data sent by the external device; SPI output interface, connected to the RK3588 processor, used to output code stream data to the RK3588 processor; Image input interface, connected to the RK3588 processor, used to receive image data decoded by the RK3588 processor; Read and write interface, used to read and write image data decoded by RK3588 processor; A PAL output interface, connected to an external device through an interface unit, for sending stored image data to the external device; PAL input interface, used for receiving PAL input signal and providing synchronous clock and control signal; The output end of the RK3588 processor is connected to the FPGA chip through a multi-channel PCIE interface, and is used to send the decoded image data to the FPGA chip.

[0007] Furthermore, the FPGA chip further includes: Synchronous output serial port: connected to external devices through the interface unit, used to forward the image data received after being decoded by the RK3588 processor; GPIO interface, connected to the RK3588 processor, used for interruption and reset of RK3588 reading the code stream data in the FIFO buffer via the SPI interface.

[0008] Furthermore, the interface unit includes a connector XS1, which is an internal data port of the image decoding unit and is used to power the image decoding unit; the connector XS1 is connected to a power supply system; the output end of the external device is connected to at least two SPI input interfaces of the FPGA chip via the connector XS1; the connector XS1 is also connected to a UART interface of the FPGA chip for debugging; the FPGA chip also outputs to the RS422 conversion chip via at least two synchronous serial ports, and the synchronous RS422 level is achieved through the RS422 conversion chip and connected to the external device via the connector XS1 to forward the received image data decoded by the RK3588 processor.

[0009] Further, the interface unit includes connectors XS2~XS4; the connectors XS2 and XS3 are analog video output ports, and the connector XS4 is a synchronization signal input port; the output end of the RK3588 processor is connected to the FPGA chip through a BT656 interface; the FPGA chip is connected to the connector XS2 via a first conversion chip, and is also connected to the connector XS3 via a second conversion chip, and the first conversion chip and the second conversion chip are used to convert the RGB signal format into a PAL signal and send it to an external device; the connector XS4 is connected to the input end of the FPGA chip via a third conversion chip, and the third conversion chip is used to convert the external PAL signal into a BT656 signal format and send it to the FPGA chip.

[0010] Furthermore, the interface unit includes a connector XS5, which is a program upgrade port of the image decoding unit and is used for program debugging and upgrading. The RK3588 processor and the FPGA chip are electrically connected to the connector XS5 through various debugging interfaces; a Gigabit Ethernet is also connected between the RGMII port of the RK3588 processor and the connector XS5, which is used as a debugging network port.

[0011] A working method based on a nationalized image decoding unit of the present invention comprises the following steps: S1: FPGA receives the bit stream data transmitted by the external device through at least one communication interface, stores it in the internal buffer after processing, and sends the bit stream data to be processed in the buffer to the RK3588 processor after receiving the instruction; S2: The RK3588 processor receives the code stream data, decodes it, and sends the decoded image data back to the FPGA for storage; S3: FPGA determines whether an external PAL input signal is received. If a PAL input signal is received, FPGA reads the stored image data based on the PAL input signal and outputs it. If no PAL input signal is received, FPGA simulates the PAL input signal timing, reads the stored image data and outputs it.

[0012] Further, in S1, the process of sending and receiving code stream data of the FPGA includes: S1-1: FPGA receives serial code stream data transmitted by an external device through at least one serial communication interface; S1-2: The FPGA uses the clock signal of the serial communication interface to perform real-time sampling on the received serial code stream data, and converts the sampled serial code stream data into parallel code stream data, and stores the data into an internal buffer area; S1-3: After receiving the control command sent by the RK3588 processor, the FPGA returns the valid length information of the code stream data to be processed in the buffer area to the RK3588 processor; S1-4: The RK3588 processor reads the code stream data of the corresponding length by controlling the clock signal according to the length of the received data, and decodes the received code stream data into original image data to complete the data processing.

[0013] Furthermore, in S1 and S2, the FPGA receives at least two SPI serial data streams from external devices, stores them in the FIFO buffer after processing, and forwards the stream data in the FIFO buffer to the RK3588 processor through the SPI interface after receiving the control command sent by the RK3588 processor; the RK3588 processor performs data decoding after receiving the stream data, and transmits the decoded image data back to the FPGA through the PCIE interface in H2C mode; after receiving the image data transmitted by the RK3588 processor, the FPGA stores it in its internal memory.

[0014] Furthermore, in S2, the process of FPGA receiving image data includes: before performing the read and write operation of the memory, FPGA caches the previous level data through the FIFO buffer; then uses the ping-pong operation mode to read and write the memory, and when writing data, selects a different bank from the current write operation to read data to avoid read and write conflicts.

[0015] Further, S3 specifically includes the following steps: the FPGA determines whether an external PAL input signal is received; if a PAL input signal is received, based on the timing of the PAL input signal, the image data is read from the memory, and the image data is output through at least two PAL output signals; if no PAL input signal is received, the FPGA simulates the timing of the PAL input signal, reads the image data from the memory according to the simulated PAL timing, and outputs the image data through at least two PAL output signals.

[0016] Beneficial effects of the present invention: (1) The combination of the domestically produced RK3588 processor and FPGA chip can perform multi-channel video decoding; (2) External video output PAL interface, capable of special technical compatibility design for analog and digital displays; (3) Using FPGA as clock intermediary, direct data transmission of PCIE interface, application of ping-pong operation on DDR3 and PAL signal synchronization technology jointly optimize data flow and image output, reduce processing delay and ensure low-latency system requirements; (4) By setting up multiple interfaces, more video processing can be achieved; for example: video watermark overlay, actual image resolution information output, Y byte component extraction, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of the circuit principle of the image decoding unit according to an embodiment of the present invention; Figure 2 This is a simplified schematic diagram of the working method of the image decoding unit according to an embodiment of the present invention; Figure 3 It is a structural block diagram of the FPGA chip according to an embodiment of the present invention; Figure 4 It is a schematic diagram of the code stream receiving and sending process of the FPGA chip according to an embodiment of the present invention; Figure 5 is a schematic diagram of a process of receiving image data by FPGA according to an embodiment of the present invention; Figure 6 is a schematic diagram of a process of receiving image data by PAL according to an embodiment of the present invention; Figure 7 It is a schematic diagram of the specific output timing of PAL according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] like Figure 1 As shown: an image decoding unit based on national production, including a processing unit, an interface unit and a power supply system; the processing unit includes a domestic RK3588 processor and an FPGA chip; the interface unit includes connectors XS1~XS5; the power supply system is used to supply power to the RK3588 processor and the FPGA chip.

[0020] In this embodiment, the RK3588 processor is a high-end general-purpose SOC using an 8nm process technology. The CPU adopts an octa-core architecture of quad-core A76 and quad-core A55. The GPU (image processing unit) adopts Mail G610 MP4, integrates a 6TOPS independent NPU, and supports four-channel LPDDR4 / 5, which effectively guarantees the performance output of the RK3588 chip. RK3588 is also very outstanding in video encoding and decoding, supporting hard encoding and decoding of 8K videos, and supports up to 8K@60fps decoding and output, which can provide stable and high-definition images for VR applications. RK3588 uses an independent hardware decoder, which has higher decoding efficiency, can effectively reduce power consumption, will not occupy CPU resources, and the CPU can continue to process other applications, ensuring performance. In addition, RK3588 supports the latest LPDDR5 memory particles, and the running memory adopts a four-channel design, which can fully guarantee the performance of the RK3588 platform. The powerful computing power of RK3588 itself can be used to post-process the data collected by the camera to achieve image optimization processing, splicing processing, superposition processing, etc.

[0021] In this embodiment, according to the technical protocol requirements of the image decoding unit, the synchronous input interface and the decompressed video output interface are both PAL signals, but the video output interface of RK3588 cannot support the requirements in the current technical protocol. Based on the above reasons, this embodiment uses FPGA chips to complete the video data format conversion and interface design. Comprehensively evaluate the logic resources required to implement the interface requirements in the technical protocol and the localization requirements, and select Fudan Micro's JFM7K325T-N as the FPGA chip of the video display processing board.

[0022] The main interface requirements of the FPGA chip in this embodiment are as follows: 1) Two SPI input interfaces: used to receive the code stream data sent by external devices and store it in the FIFO of the FPGA chip; 2) Two SPI output interfaces: used to enable the RK3588 processor to read the code stream data stored in the FIFO; 3) Two synchronous serial port outputs: used to forward the image data received after being decoded by the RK3588 processor; 4) Two MIPI input interfaces: used to receive image data decoded by the RK3588 processor; 5) Standard DDR3 read / write interface: used to read and write image data decoded by RK3588 processor; 6) Two PAL output interfaces: used to send image data stored in DDR3 to external devices; 7) One PAL input interface: used to provide the clock and control signal of PAL output timing; 8) Four GPIO interfaces: used for interruption and reset of two RK3588s reading FIFO code stream data via SPI.

[0023] In this embodiment, the specific connection structure of the RK3588 processor and the FPGA chip is as follows: the two SPI output interfaces of the FPGA chip are connected to the input end of the RK3588 processor, so that the RK3588 processor can read the code stream data stored in the FIFO; the four GPIO interfaces of the FPGA chip are connected to the RK3588 processor; the FPGA chip communicates with the RK3588 processor through the I2C bus; the UART interface of the FPGA chip is connected to the UART interface of the RK3588 processor for serial communication; the RK3588 processor is connected to the FPGA chip through four PCIE interfaces, and the decoded image data is logically divided into two paths by using the Host-to-Card (H2C) method of PCIE, and transmitted in parallel to the FPGA for processing. The output end of the RK3588 processor is connected to the FPGA chip through the BT656 interface for transmitting video data for subsequent processing. The MIPI interface of the RK3588 processor is connected to the FPGA chip for sending the decoded image data to the FPGA chip.

[0024] The above connection structure reflects the close cooperation between the FPGA chip and the RK3588 processor through multiple communication interfaces, aiming to provide efficient data transmission and processing capabilities. Each interface plays a unique role and cooperates with each other to ensure the high performance and stability of the system. Specifically, the SPI interface provides high-speed serial communication, which can efficiently transfer the code stream data from the FPGA's FIFO to the RK3588 processor. The use of two SPI interfaces increases the bandwidth of data transmission and ensures fast and stable reading of data. The GPIO interface provides a flexible signal control method, which can be used for triggering, status monitoring and control signal transmission. Through four GPIOs, low-latency control and status exchange can be performed between the FPGA and the RK3588 processor; through the I2C bus, the FPGA and the RK3588 processor can realize parameter configuration, device initialization and other functions; UART is used for serial data communication, which is suitable for low-speed, reliable serial transmission. In this design, the UART connection between the FPGA and the RK3588 can be used for system-level debugging, log transmission or low-speed data exchange; PCIE provides high-bandwidth, high-throughput data transmission capabilities, especially for applications that require large amounts of data to be processed in parallel. Through the PCIE interface, the RK3588 can divide the decoded image data into two parallel transmissions to the FPGA in a Host-to-Card manner, thereby greatly improving the system's processing efficiency and data throughput. BT656 is a standard video transmission protocol, which is widely used in video decoding and display devices. Through the BT656 interface, the RK3588 can efficiently transmit the decoded image data to the FPGA, which is suitable for application scenarios that process video data. MIPI is a high-speed serial interface. RK3588 can transmit image data to FPGA through the MIPI interface.

[0025] In this embodiment, the RK3588 processor can also directly connect to the HDMI device via the HDMI interface via the HDMI connector.

[0026] In this embodiment, connector XS1 is the internal data port of the image decoding unit and is used to power the image decoding unit. Specifically, the power supply system powers the image decoding unit via connector XS1, adopts +5V DC input, and generates the voltage required by each device through a DC / DC conversion chip or an LDO chip. In addition, the output end of the external device is connected to the two SPI input interfaces of the FPGA chip via connector XS1, which is used to send code stream data to the FPGA chip; connector XS1 is connected to one UART interface of the FPGA chip for debugging; the FPGA chip also outputs to the conversion chip via two synchronous serial ports, and realizes synchronous RS422 level through the conversion chip, and is connected to the external device via connector XS1 to forward the received image data decoded by the RK3588 processor. The conversion chip is preferably the HR3490 chip of Beijing Huaxinwei.

[0027] In this embodiment, connectors XS2 and XS3 are analog video output ports, and connector XS4 is a synchronization signal input port. Among them, the synchronization signal input port and the analog video output port are both PAL signals, and the FPGA chip is connected to connector XS2 via the first conversion chip to facilitate communication with the corresponding external device. The FPGA chip is connected to connector XS3 via the second conversion chip; when the FPGA chip receives the image data of the RK3588 processor, it is connected to each conversion chip respectively, so as to be converted into the analog signal output required by the external device, and the corresponding external device can be a display. Among them, the first conversion chip and the second conversion chip both use the GM7392 chip, which is used to convert the RGB signal format into a PAL signal and send it to the display. Connector XS4 is connected to the input end of the FPGA chip via the third conversion chip, and the third conversion chip is used to convert the external PAL signal into the BT656 signal format and send it to the FPGA chip; the third conversion chip uses Chengdu Zhenxin's GM7150A.

[0028] In this embodiment, connector XS5 is the program upgrade port of the image decoding unit, which is used for program debugging and upgrading. Specifically, the RK3588 processor and the FPGA chip are electrically connected to the connector XS5 through various debugging interfaces. For example: a Gigabit Ethernet is connected between the RGMII port of the RK3588 processor and the connector XS5, which is used as a debugging network port, and can realize external communication through the 100Mbps Ethernet interface. The RK3588 processor is connected to the connector XS5 line through a dedicated USB download interface, and the system or UBOOT can be upgraded. The external PC can download the file system or IMG file required by the RK3588 processor, which is convenient for customers to upgrade. The RK3588 processor prints the system startup log through a dedicated UART debugging interface, and the external PC can observe the startup process of some RK3588 systems, which is convenient for customers to debug. For another example: the FPGA chip is electrically connected to the connector XS5 through the JTAG debugging port to provide debugging access.

[0029] In this embodiment, both the RK3588 processor and the FPGA chip are equipped with memory chips. Among them, the RK3588 processor provides a 2-way 32-bit LPDDR4x memory controller interface. Combined with the board size and technical requirements, a 32-bit LPDDR4x is selected. Each channel only requires 1 32-bit memory chip, and a total of 2 memory chips are used to form 2 32-bit dual-channel 8GB memory capacity. The memory controller bandwidth can reach 29.8GB / s. The memory chip uses a board-mounted type to replace the traditional memory slot plus memory bar method to enhance the board's ability to resist vibration and impact. The FPGA chip is equipped with 1GB of DDR3 memory for video data cache, and the model is preferably 2 pieces of XZF41J256M16 from Zhongtianxing Control.

[0030] The RK3588 processor and FPGA chip are also equipped with storage chips. Among them, the RK3588 storage end is equipped with 32GB EMMC, which is used to store file systems, operating systems and upgradeable programs. The FPGA chip is configured with NOR Flash, specifically using 1 EFM25F128A to form a 16MB capacity, which is used to store FPGA configuration files.

[0031] like Figure 2 and Figure 3As shown, the working method of this embodiment is as follows: first, the FPGA receives two SPI code stream data and forwards it to the RK3588 processor through the SPI interface. The RK3588 processor decodes the received code stream and transmits the image data to the FPGA chip through the PCIE interface in two H2Cs. After receiving the image data, the FPGA stores it in the DDR3 memory. Next, the FPGA determines whether there is an external PAL_IN signal input. If there is a PAL_IN signal input, the FPGA will read the stored image data from the DDR3 memory according to the timing of the PAL_IN signal, and output it through the two PAL_OUT interfaces. If there is no PAL_IN signal input, the FPGA simulates the PAL_IN timing and outputs the image data stored in the DDR3 memory through the two PAL_OUT interfaces.

[0032] The above technology ensures the compatibility design of the system. On the one hand, when the system receives an external PAL_IN signal, the FPGA will synchronize the image output according to the timing of this signal. This ensures that the image data is displayed according to the standard PAL timing, avoiding image delay or distortion caused by asynchronous timing. On the other hand, when there is no external PAL_IN signal, the FPGA simulates the timing of the PAL_IN signal to output the image, further enhancing the compatibility of the system. This means that even without an external synchronization signal, the system can still output image data that meets the PAL standard, thereby ensuring that the device can work stably in various environments and enhancing the versatility and compatibility of the system.

[0033] like Figure 4 As shown in the figure: In this embodiment, the FPGA code stream receiving and sending process is as follows: the FPGA chip first receives the serial code stream data transmitted by the SPI interface, and then samples the data using the rising edge of the SPI clock, converts the sampled serial code stream data into parallel code stream data and stores it in the FIFO buffer. Next, after receiving the control command of the RK3588 processor, the FPGA returns the length of the data to be processed in the FIFO to RK3588. After obtaining the data length, RK3588 reads the code stream data of the corresponding length by controlling the clock signal, and decodes the received code stream data into the original image to complete the data processing.

[0034] The reason why we choose to use FPGA as an intermediary instead of directly transmitting SPI data to the RK3588 processor is that the code rate of the external SPI input is not fixed, and the SPI clock rate may fluctuate, causing the RK3588 to be unable to effectively process the variable-rate SPI signal. FPGA as an intermediary can solve the problem of unstable clock rate and ensure the stability of data transmission and decoding.

[0035] like Figure 5As shown: In this embodiment, the process of FPGA receiving image data is as follows: After FPGA receives PCIE data, it writes the data into DDR3 memory. Before reading and writing DDR3, the previous stage will use FIFO buffer to avoid simultaneous reading and writing operations to ensure data consistency. In order to ensure the integrity of the image and solve the image tearing problem, FPGA uses ping-pong operation to read and write DDR3. In this way, when writing data (for example, writing to Bank1), data can only be read from another Bank (for example, Bank2), thereby avoiding read-write conflicts.

[0036] Data is transmitted between RK3588 and FPGA via PCIE, which can directly access the memory of FPGA. Therefore, the decoded data does not need additional cache or cross-clock domain processing, and is directly sent to FPGA. Since the frame rate of PAL signal is 50 frames per second and interlaced scanning is used, it takes 40ms to transmit one frame, which can effectively reduce transmission delay and meet the needs of low-latency applications.

[0037] like Figure 6 As shown in the figure: In this embodiment, the process of PAL receiving image data is as follows: FPGA first determines whether there is an external PAL_IN signal input. If the PAL_IN signal input is detected, the image data stored in DDR3 is output through two PAL_OUTs according to the timing of PAL_IN; if there is no PAL_IN signal input, FPGA will simulate the PAL_IN timing and output the image data stored in DDR3 through two PAL_OUTs.

[0038] The synchronization function of the PAL_IN signal is mainly used for image overlay. When the output PAL screen needs to be overlaid, other external images can be easily and synchronously overlaid on the current screen without the need for additional complex algorithm processing, which helps to simplify system design and improve system stability. If external synchronization is not required, the PAL_IN signal can be disconnected. The FPGA will automatically detect that there is no PAL_IN input and output image data by simulating the PAL_IN timing.

[0039] The specific output timing of PAL is as follows Figure 7 As shown in the figure, the PAL output adopts the standard BT656 data format, with a resolution of 720*576, a frame rate of 50bps, and an interlaced scanning mode.

[0040] In summary, the present invention has the following advantages: (1) Efficient video decoding and processing capabilities: Using the domestically produced RK3588 processor for two-way video decoding means that the system has the ability to decode in parallel, which enables the system to process video streams from different sources at the same time and ensure efficient data decoding; the RK3588 processor usually has strong multi-core processing capabilities, suitable for processing high-concurrency video data streams and improving the overall performance of the system; in addition to the decoding function, the RK3588 processor can also be combined with FPGA to implement more video processing tasks, such as video watermark overlay, resolution conversion, color adjustment, etc. These processing functions can be applied to the decoded video stream in real time to provide higher quality output; (2) PAL interface output and compatibility design The design of the PAL interface enables the system to support both traditional analog displays and digital displays. Through the compatibility of hardware design, it can automatically adapt to the input requirements of different display devices without the need for additional device conversion. This technical compatibility design greatly simplifies the system interface design and expands the scope of application of the device. ② Signal format conversion: The video output is automatically converted into an analog signal that complies with the PAL standard through each conversion chip, and high-quality digital video output can also be provided to ensure that the display device can correctly display the decoded video stream; (3) FPGA as clock intermediary, direct data transmission of PCIE interface, application of ping-pong operation on DDR3 and PAL signal synchronization technology jointly optimize data flow and image output, reduce processing delay and ensure low-latency system requirements, specifically: ① FPGA as clock intermediary solves the problem of SPI clock instability: After receiving the SPI code stream, FPGA uses its internal stable clock to sample and convert the data, and forwards it to the RK3588 processor with stable timing. In this way, FPGA effectively solves the timing problem caused by SPI clock fluctuation, thereby ensuring the stability of data flow and reducing transmission delay. ② Data transmission optimization of PCIE interface: RK3588 processor transmits decoded image data between FPGA and PCIE interface. PCIE has high-speed transmission capability and can directly access the memory of FPGA, avoiding additional cache or data transmission across clock domains. Therefore, image data can be directly transferred from RK3588 to FPGA for processing without complex buffering or additional synchronization mechanism, reducing intermediate steps and unnecessary delays. ③Use ping-pong operation to avoid DDR3 read and write conflicts: In order to ensure data consistency and avoid data tearing, FPGA uses ping-pong operation to read and write DDR3. In this mode, when data is written to a DDR3 storage bank, data reading occurs in another bank. This method effectively avoids read and write conflicts, ensures the integrity of image data, and improves processing speed through parallel read and write operations, reducing delays caused by waiting and synchronization. ④PAL signal frame rate and interlaced scanning technology: When outputting PAL signals, the image frame rate is 50 frames per second, and the transmission time of each frame is 40ms, which can effectively reduce transmission delays. Using interlaced scanning technology, it only takes 40ms to transmit each frame of the picture. Compared with the line-by-line scanning method, the output of image data can be completed faster, thus meeting the application requirements of low latency. ⑤FPGA synchronizes image output according to PAL_IN signal: When FPGA detects the external PAL_IN signal, it will synchronize the image output according to the timing of the PAL_IN signal, thereby avoiding delays and image tearing caused by image asynchrony. When there is no PAL_IN signal, the FPGA will output the image by simulating the timing of the PAL_IN signal, further optimizing the delay performance and ensuring that the system can output image data with stable timing even when there is no external synchronization signal.

[0041] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A nationally produced image decoding unit, comprising a processing unit, an interface unit and a power supply system; characterized in that: The processing unit includes a domestically produced RK3588 processor and an FPGA chip; The FPGA chip comprises: SPI input interface, connected to an external device through an interface unit, for receiving code stream data sent by the external device; SPI output interface, connected to the RK3588 processor, used to output code stream data to the RK3588 processor; Image input interface, connected to the RK3588 processor, used to receive image data decoded by the RK3588 processor; Read and write interface, used to read and write image data decoded by RK3588 processor; A PAL output interface, connected to an external device through an interface unit, for sending stored image data to the external device; PAL input interface, used for receiving PAL input signal and providing synchronous clock and control signal; The output end of the RK3588 processor is connected to the FPGA chip through a multi-channel PCIE interface, and is used to send the decoded image data to the FPGA chip.

2. The nationalized image decoding unit according to claim 1, characterized in that: The FPGA chip also includes: Synchronous output serial port: connected to external devices through the interface unit, used to forward the image data received after being decoded by the RK3588 processor; GPIO interface, connected to the RK3588 processor, used for interruption and reset of RK3588 reading the code stream data in the FIFO buffer via the SPI interface.

3. The nationalized image decoding unit according to claim 1 or 2, characterized in that: The interface unit includes a connector XS1, which is the internal data port of the image decoding unit and is used to power the image decoding unit; the connector XS1 is connected to the power supply system; the output end of the external device is connected to at least two SPI input interfaces of the FPGA chip via the connector XS1; the connector XS1 is also connected to a UART interface of the FPGA chip for debugging; the FPGA chip also outputs to the RS422 conversion chip via at least two synchronous serial ports, and the synchronous RS422 level is achieved through the RS422 conversion chip and connected to the external device via the connector XS1 to forward the received image data decoded by the RK3588 processor.

4. The nationalized image decoding unit according to claim 1 or 2, characterized in that: The interface unit includes connectors XS2~XS4; the connectors XS2 and XS3 are analog video output ports, and the connector XS4 is a synchronization signal input port; the output end of the RK3588 processor is connected to the FPGA chip through a BT656 interface; the FPGA chip is connected to the connector XS2 via a first conversion chip, and is also connected to the connector XS3 via a second conversion chip, and the first conversion chip and the second conversion chip are used to convert the RGB signal format into a PAL signal and send it to an external device; the connector XS4 is connected to the input end of the FPGA chip via a third conversion chip, and the third conversion chip is used to convert the external PAL signal into a BT656 signal format and send it to the FPGA chip.

5. The nationalized image decoding unit according to claim 1 or 2, characterized in that: The interface unit includes a connector XS5, which is a program upgrade port of the image decoding unit and is used for program debugging and upgrading. The RK3588 processor and the FPGA chip are electrically connected to the connector XS5 through various debugging interfaces. A Gigabit Ethernet is also connected between the RGMII port of the RK3588 processor and the connector XS5, which is used as a debugging network port.

6. A working method based on a nationalized image decoding unit, characterized in that: The following steps are involved: S1: FPGA receives the bit stream data transmitted by the external device through at least one communication interface, stores it in the internal buffer after processing, and sends the bit stream data to be processed in the buffer to the RK3588 processor after receiving the instruction; S2: The RK3588 processor receives the code stream data, decodes it, and sends the decoded image data back to the FPGA for storage; S3: FPGA determines whether an external PAL input signal is received. If a PAL input signal is received, FPGA reads the stored image data based on the PAL input signal and outputs it. If no PAL input signal is received, FPGA simulates the PAL input signal timing, reads the stored image data and outputs it.

7. The working method based on the nationalized image decoding unit according to claim 6 is characterized in that: In S1, the process of sending and receiving code stream data of the FPGA includes: S1-1: FPGA receives serial code stream data transmitted by an external device through at least one serial communication interface; S1-2: The FPGA uses the clock signal of the serial communication interface to perform real-time sampling on the received serial code stream data, and converts the sampled serial code stream data into parallel code stream data, and stores the data into an internal buffer area; S1-3: After receiving the control command sent by the RK3588 processor, the FPGA returns the valid length information of the code stream data to be processed in the buffer area to the RK3588 processor; S1-4: The RK3588 processor reads the code stream data of the corresponding length by controlling the clock signal according to the length of the received data, and decodes the received code stream data into original image data to complete the data processing.

8. The working method based on the nationalized image decoding unit according to claim 6 or 7, characterized in that: In S1 and S2, the FPGA receives at least two SPI serial data streams from external devices, stores them in the FIFO buffer after processing, and forwards the stream data in the FIFO buffer to the RK3588 processor through the SPI interface after receiving the control command sent by the RK3588 processor; the RK3588 processor decodes the data after receiving the stream data, and transmits the decoded image data back to the FPGA through the PCIE interface in H2C mode; after receiving the image data transmitted by the RK3588 processor, the FPGA stores it in its internal memory.

9. The working method based on the nationalized image decoding unit according to claim 8, characterized in that: In S2, the process of FPGA receiving image data includes: before performing the read and write operation of the memory, FPGA caches the previous level data through the FIFO buffer; then uses the ping-pong operation method to read and write the memory. When writing data, selects a different bank from the current write operation to read data to avoid read and write conflicts.

10. The working method based on the nationalized image decoding unit according to claim 8, characterized in that: S3 specifically includes the following steps: the FPGA determines whether an external PAL input signal is received; if a PAL input signal is received, the FPGA reads image data from a memory based on the timing of the PAL input signal, and outputs the image data through at least two PAL output signals; if no PAL input signal is received, the FPGA simulates the timing of the PAL input signal, reads image data from the memory according to the simulated PAL timing, and outputs the image data through at least two PAL output signals.

Images