8K / 4K ultra-high-definition native IP media signal format conversion device and conversion method
By combining an embedded FPGA+ARM architecture with HBM memory, the problem of the lack of 8K ultra-high-definition signal format conversion equipment has been solved, achieving efficient and low-cost signal conversion to meet real-time processing requirements.
Patent Information
- Application Number
- CN202411109109.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-22
AI Technical Summary
There is a lack of domestic 8K ultra-high-definition signal format conversion equipment. Existing server solutions cannot meet real-time requirements in terms of processing speed and are costly, making them unsuitable for multi-channel deployment.
The system adopts an embedded FPGA+ARM architecture. The FPGA in the signal processing module completes the encapsulation and decapsulation of IP signal data, while the ARM in the signal control module completes the parameter configuration. Combined with HBM memory and high-performance FPGA chip, it realizes uncompressed conversion of 8K/4K signal format and meets high bandwidth requirements by expanding the on-chip cache read and write bandwidth.
It achieves miniaturized, low-power, and cost-effective 8K/4K signal format conversion, supports 2-channel real-time conversion, has high-efficiency signal conversion density and multiple processing capabilities, and fills a gap in domestic technology.
Smart Images

Figure CN119011945A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of communication technology and broadcasting and television technology, and in particular to an 8K / 4K ultra-high-definition native IP media signal format conversion device and conversion method. Background Art
[0002] Under the background of the continuous development of ultra-high-definition technology, the demand for up-down conversion of signals in various links of the broadcasting process will continue to increase. In the process of up-down conversion of signal formats, in addition to resolution, there are also key technical indicators such as frame rate, color gamut, dynamic range, sampling rate, bit depth, and audio. At present, foreign IP domain format conversion equipment mainly includes 8K Cross Converter (SC-8229), 8K 120Hz Converter (SC-8221), 8K U-SDI to 12G-SDI Converter (SC-8221) from Japan's ASTRO, HDRC-4000 from Japan's SONY, Kudos Pro and UHD1200 from GV in the United States, Teranex AV from Blackmagic Design in Australia, FS4 from AJA International in the United States, MCC-4K-A from FOR-A and other products.
[0003] The domestic ultra-high-definition television industry is in its infancy, and the core technologies and products of high-quality ultra-high-definition television signal conversion and processing are still monopolized by international traditional broadcasting and television equipment providers. At present, Beijing Gefei Company has a 4K ultra-high-definition IP signal processor that can realize the mutual conversion between HD and 4K signals. The native IP domain format conversion equipment in the 8K ultra-high-definition field is still blank.
[0004] In the field of ultra-high-definition production, some domestic program production and packaging manufacturers use server-based technology to achieve 8K ultra-high-definition signal format conversion. The implementation process is: with the help of imported ultra-high-definition interface cards, installed in the server PCIE slot to achieve signal input and output; with the help of CPG+GPU, multi-threading is enabled to complete 8K high-speed signal conversion.
[0005] The disadvantages of the above solution are:
[0006] 1. The server processing speed cannot meet the real-time signal processing requirements and cannot be directly connected to the production and broadcasting transmission network;
[0007] 2. The server cost of implementing the above functions is relatively high and is not suitable for multi-channel deployment.
[0008] In order to meet the demand for localization of key equipment, fill the gap in this field in China, and get rid of the situation of relying on imported equipment, it is urgent to develop a device that can convert 8K signals / 4K format uncompressed digital signals into each other. Summary of the invention
[0009] In view of the current insufficiency of server solutions and the lack of solutions on embedded FPGA platforms, the present invention provides an 8K / 4K ultra-high-definition native IP media signal format conversion device and conversion method, which realizes the mutual conversion of 8K signal / 4K format uncompressed digital signal based on an embedded platform.
[0010] The present invention discloses an 8K / 4K ultra-high definition native IP media signal format conversion device, which is characterized by:
[0011] It includes: interconnected signal processing module, signal control module and power supply module. The FPGA of the signal processing module completes IP signal data packaging and depacketization as well as signal conversion processing. The ARM of the signal control module completes communication with the upper software and configuration of working parameters.
[0012] The signal processing module includes two 100G optical fiber network interfaces, a network protocol unpacking module and a network protocol packaging module, an ST2110 protocol unpacking module and an ST2110 protocol packaging module, a PTP clock module, an HBM memory, a signal up / down conversion processing module and a configuration interface; the network protocol unpacking module and the network protocol packaging module are both connected to the 100G optical fiber network interface, the ST2110 protocol unpacking module and the ST2110 protocol packaging module, and the PTP clock module, the ST2110 protocol unpacking module and the ST2110 protocol packaging module are connected to the HBM memory, the HBM memory is connected to the signal up / down conversion processing module, and the FPGA of the signal processing module is connected to the ARM of the signal control module through the configuration interface;
[0013] The signal control module includes an interface, a protocol stack and a client Web configuration page which are connected in sequence.
[0014] As a further improvement of the present invention, the 100G optical fiber network interface is a QSFP28 optical fiber interface.
[0015] The present invention also discloses an 8K / 4K ultra-high-definition native IP media signal format conversion method based on the above-mentioned 8K / 4K ultra-high-definition native IP media signal format conversion device, comprising:
[0016] The external core switch transmits the 8K / 4K IP signal to the signal processing module via the 100G optical fiber network interface. In the signal processing module, the signal is first demultiplexed by the network protocol depacketization module and then sent to the ST2110 protocol depacketization module. The video, audio, and auxiliary data after depacketization based on the PTP clock module are cached in the HBM memory.
[0017] The video, audio, and auxiliary data in the HBM memory are extracted to the signal up / down conversion processing module, and the signal up / down conversion is completed according to the user configuration requirements provided by the signal control module. The converted data is sent to the HBM memory; the signal up / down conversion includes: signal frame synchronization, 8K / 4K image resolution conversion, 8K / 4K video brightness curve, WCG color gamut conversion, and 8K / 4K signal frame rate conversion;
[0018] The converted data in the HBM memory is sent to the ST2110 protocol packet module, and the uncompressed ST2110 protocol IP data packet is completed based on the PTP clock module. Then the packetized data is sent to the network protocol packet module for network multiplexing, and finally the processed IP stream is sent back to the external core switch via the 100G optical fiber network interface;
[0019] The signal control module provides the device with an external interface communication protocol through a protocol stack, and provides a client web configuration interface for users to modify working parameters and view working status.
[0020] As a further improvement of the present invention, in the HBM memory, a method for extending the read and write bandwidth of the on-chip cache is used to implement writing and reading of ultra-high bandwidth data, specifically including:
[0021] When controlling access, the FPGA clock based on the signal processing module generates a timing control signal for the HBM;
[0022] Assign values to the AXI-ports. The corresponding address channels and HBM channels under each AXI-port have corresponding addresses to construct a storage matrix containing M AXI-ports.
[0023] When data is written, the data is evenly divided and written into the storage matrix from M AXI-ports; when data needs to be read, it is read from multiple AXI-ports at the same time.
[0024] As a further improvement of the present invention, when writing data to HBM through AXI-port, a new address algorithm with the same address bits and different data bits is adopted. The new address consists of a particle ID and a particle access address, and the algorithm controls the writing and reading of data to the new address.
[0025] Compared with the prior art, the present invention has the following beneficial effects:
[0026] The present invention adopts an embedded FPGA+ARM architecture. The FPGA of the signal processing module completes IP signal data packaging and depacketization as well as signal conversion processing; the ARM of the signal control module completes communication with the upper-layer software and configuration of working parameters; it has the characteristics of small size (can be integrated in a 1RU rack chassis or a half-width 1RU chassis to save rack space), low power consumption, portability, and high cost performance;
[0027] The present invention is based on a high-performance, multifunctional FPGA high-speed integrated processing chip. The single device supports real-time conversion of two-way 8K signals. The two channels can complete conversion in different directions (8K to 4K or 4K to 8K), and are automatically compatible with 3G / 1.5G format signals, improving signal conversion density and efficiency in a smaller space.
[0028] The present invention utilizes the same address bit characteristic of HBM particles, and adopts a processing mechanism of same address bit but different data bit when writing data to HBM through AXI port, and utilizes a new address combination algorithm, the new address includes HBM particle ID and HBM particle access address, the new algorithm controls ultra-high bandwidth data writing and reading, and realizes expansion of read and write bandwidth; meets the demand of single device for real-time and multiple processing of ultra-high bandwidth and high-speed signals that are inseparable from high-resolution images; based on this application scheme, single embedded device provides multiple signal conversion processing;
[0029] The hardware architecture and functional design of the present invention are in a leading position both domestically and internationally, filling the gaps in domestic technology and equipment. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a structural schematic diagram of the 8K / 4K ultra-high-definition native IP media signal format conversion device disclosed in the present invention;
[0031] Figure 2 It is a schematic diagram of the I / O bandwidth of the existing HBM;
[0032] Figure 3 A schematic diagram of the I / O bandwidth of an HBM memory disclosed in an embodiment of the present invention. DETAILED DESCRIPTION
[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] The present invention is further described in detail below in conjunction with the accompanying drawings:
[0035] The present invention provides an 8K / 4K ultra-high-definition native IP media signal format conversion device. The device hardware is based on FPGA+ARM platform. FPGA has high programmability and parallel computing capabilities, and can be used for real-time data processing and stream analysis; the ARM processor is used for device working parameter configuration and data communication with upper-layer software; the present invention performs data input and output through a 100G optical fiber network interface, performs 8K / 4K IP media stream data caching and conversion processing through FPGA, and the ARM chip provides configuration and management functions for device conversion; selecting a combination of FPGA and ARM architecture can give full play to the advantages of both.
[0036] Specific:
[0037] like Figure 1 As shown, the 8K / 4K ultra-high-definition native IP media signal format conversion device of the present invention includes: an interconnected signal processing module, a signal control module and a power supply module; wherein,
[0038] The signal processing module includes two 100G optical network interfaces, a network protocol unpacking module and a network protocol encapsulating module, an ST2110 protocol unpacking module and an ST2110 protocol encapsulating module, a PTP clock module, an HBM memory, a signal up / down conversion processing module, and a configuration interface; the network protocol unpacking module and the network protocol encapsulating module are both connected to the 100G optical network interface, the ST2110 protocol unpacking module and the ST2110 protocol encapsulating module, and the PTP clock module, the ST2110 protocol unpacking module and the ST2110 protocol encapsulating module are connected to the HBM memory, the HBM memory is connected to the signal up / down conversion processing module, and the signal processing module is connected to the signal control module through the configuration interface;
[0039] The signal control module includes an interface, a protocol stack, and a client web configuration page that are connected in sequence.
[0040] The present invention provides an 8K / 4K ultra-high-definition native IP media signal format conversion method based on the above-mentioned 8K / 4K ultra-high-definition native IP media signal format conversion device, comprising:
[0041] S1. The external core switch transmits the 8K / 4K IP signal (8K / 4K IP media stream data) to the FPGA via the 100G optical network interface (QSFP28 optical interface); in the signal processing module, it is first demultiplexed by the network protocol depacketization module, and then sent to the ST2110 protocol depacketization module. The video, audio, and auxiliary data (unpacked 8K / 4K IP media stream data) unpacked based on the PTP clock module are cached in the HBM memory;
[0042] S2. Extract the video, audio, and auxiliary data in the HBM memory to the signal up / down conversion processing module, complete the signal up / down conversion according to the user configuration requirements provided by the signal control module, and send the converted data to the HBM memory; wherein, the signal up / down conversion includes: signal frame synchronization, 8K / 4K image resolution conversion, 8K / 4K video brightness curve, WCG color gamut conversion, and 8K / 4K signal frame rate conversion;
[0043] S3, the converted data in the HBM memory is sent to the ST2110 protocol packet module, and the uncompressed ST2110 protocol IP data packet is completed based on the PTP clock module, and then the packetized data is sent to the network protocol packet module for network multiplexing, and finally the processed IP stream is sent back to the external core switch via the 100G optical network interface (QSFP28 optical fiber interface);
[0044] S4. The signal control module provides the device with an external interface communication protocol through a protocol stack, and provides a client web configuration interface for users to modify working parameters (including: user configuration requirements required to complete signal up and down conversion) and view working status.
[0045] in,
[0046] The present invention adopts two 100G optical fiber network interfaces to realize SMPTE2022-7 redundant link stream access, and has the main and backup backup functions required by professional IP media networks; the main and backup 4K / 8K ultra-high-definition IP stream signals pass through the protocol analysis module to complete the ultra-high-definition audio and video data unpacking and restore it to SDI original data, including video, audio, auxiliary area data and synchronization information, which are sent to the high-bandwidth HBM memory.
[0047] The present invention uses the device embedded Webserver, so that users can access the device, view the device working status, and complete the conversion parameter configuration. The upper control software controls the device to perform IP signal control scheduling, device, and signal status data acquisition through the NMOS communication protocol.
[0048] In HBM memory, HBM is mainly used for high-performance computing chips, and is mostly used to work with data center computing chips such as GPU / FPGA / ASIC. The read and write bandwidth of HBM's I / O (AXI-Port) is 256bit / s. High-performance chips usually provide a clock frequency of about 400MHz. Assuming that the frequency is 450MHz, the HBM read and write I / O is 256bit / s×450MHz=115.2Gbit / s. The I / O bandwidth provided by the existing HBM is as follows: Figure 2As shown. The existing HBM provides 115.2Gbit / s I / O bandwidth, and it takes 96Gbit / s to process the input and output of one 8K signal. This bandwidth can meet the usage requirements. However, for complex processing requirements, such as when multiple processes need to be performed simultaneously after the 8K signal is written into the cache, the bandwidth cannot meet the requirements, and the read and write bandwidth needs to be increased again.
[0049] To this end, the present invention adopts a method for expanding the read and write bandwidth of the on-chip cache to achieve the writing and reading of ultra-high bandwidth data. The method utilizes the characteristic that the address bits of the DRAM particles of HBM are the same. When writing data to the HBM through the AXI port, a processing mechanism with the same address bits and different data bits is adopted. A new address algorithm is invented. The new address consists of a particle ID and a particle access address. The algorithm controls the writing and reading of data to the new address, and binds the number of particles according to the demand; wherein each piece of data is written equally to the new address and read out when it is needed; the number of particles required is based on the bandwidth demand for the readout. Assuming that there are multiple processors reading data from the cache later, the required memory size is planned in advance, and the rest is left for other applications.
[0050] like Figure 3 As shown, the present invention divides the data of a signal into M parts, preferably equally divided into M parts, where M is a positive integer ≥ 2; and stores them on M particles, and the read and write bandwidths are expanded M times, thereby solving the problem of completing multiple processing operations on a chip for one ultra-high bandwidth, high-speed transmission signal, and achieving the result of integrating multiple functions on a single device.
[0051] Taking the 8K data, 450Ms / clock FPGA chip as an example, the data bandwidth requirement of 8K is 48Gbit / s, and the bandwidth provided by an AXI-port is 256bits×450M=115.2Gbit / s, which can meet the reading of 2 channels of 8K signals; if M AXI-ports work at the same time, M×15.2Gbit / s bandwidth can be achieved, which can adapt to the high-performance FPGA processing capabilities and give full play to the chip's capacity resources.
[0052] The specific implementation method of the present invention is:
[0053] When controlling access, a timing control signal for HBM is generated based on the FPGA clock, so that all AXI-ports work at the same timing.
[0054] Assign values to the AXI-ports. The corresponding address channels and HBM channels under each AXI-port have corresponding addresses, thereby constructing a storage matrix containing M AXI-ports.
[0055] When writing data, the data is evenly divided and written into the storage matrix from M AXI-ports; when it needs to be read, it is read from multiple AXI-ports at the same time; for example, when writing data, 8K audio and video data is evenly divided and written into the storage matrix from M AXI-ports according to the entire image; when it needs to be read, it is read from multiple AXI-ports at the same time, thereby realizing the expansion of data reading and writing bandwidth and meeting the needs of one input and multiple outputs.
[0056] The advantages of the present invention are:
[0057] The present invention adopts an embedded FPGA+ARM architecture, where FPGA completes IP signal data packaging and depacketization as well as signal conversion processing; ARM completes communication with upper-layer software and configuration of working parameters; it has the characteristics of small size (can be integrated in a 1RU rack-mounted chassis or a half-width 1RU chassis to save rack space), low power consumption, portability, and high cost performance;
[0058] The present invention is based on a high-performance, multifunctional FPGA high-speed integrated processing chip. The single device supports real-time conversion of two-channel 8K signals. The two channels can complete conversion in different directions (8K to 4K or 4K to 8K), improving signal conversion density and efficiency in a smaller space.
[0059] The present invention adopts a high-performance FPGA chip with HBM (High Bandwidth Memory) storage particles, which has the characteristics of ultra-high bandwidth IP flow encapsulation and decapsulation delay, high PTP timestamp accuracy, real-time conversion of multi-channel ultra-high-definition signals, and real-time low-code stream encoding functions; it meets the needs of single devices for real-time and multiple processing of ultra-high bandwidth and high-speed signals that are inseparable from high-resolution images; based on this application solution, it realizes that single embedded devices provide multiple signal conversion processing;
[0060] The hardware architecture and functional design of the present invention are in a leading position both domestically and internationally, filling the gaps in domestic technology and equipment.
[0061] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. An 8K / 4K ultra-high-definition native IP media signal format conversion device, characterized in that: include: The interconnected signal processing module, signal control module and power supply module, the FPGA of the signal processing module completes the IP signal data packaging and depacketization as well as the signal conversion processing, and the ARM of the signal control module completes the communication with the upper layer software and the working parameter configuration; among them, The signal processing module includes two 100G optical fiber network interfaces, a network protocol unpacking module and a network protocol packaging module, an ST2110 protocol unpacking module and an ST2110 protocol packaging module, a PTP clock module, an HBM memory, a signal up / down conversion processing module and a configuration interface; the network protocol unpacking module and the network protocol packaging module are both connected to the 100G optical fiber network interface, the ST2110 protocol unpacking module and the ST2110 protocol packaging module, and the PTP clock module, the ST2110 protocol unpacking module and the ST2110 protocol packaging module are connected to the HBM memory, the HBM memory is connected to the signal up / down conversion processing module, and the FPGA of the signal processing module is connected to the ARM of the signal control module through the configuration interface; The signal control module includes an interface, a protocol stack and a client Web configuration page which are connected in sequence.
2. The 8K / 4K ultra-high-definition native IP media signal format conversion device according to claim 1, characterized in that: The 100G optical fiber network interface is a QSFP28 optical fiber interface.
3. A method for converting the format of an 8K / 4K ultra-high-definition native IP media signal based on the 8K / 4K ultra-high-definition native IP media signal format conversion device according to claim 1 or 2, characterized in that: include: The external core switch transmits 8K / 4K IP signals to the signal processing module via the 100G optical fiber network interface; In the signal processing module, the network protocol demultiplexing module first performs network protocol demultiplexing, and then sends it to the ST2110 protocol depacketization module. The video, audio, and auxiliary data after depacketization based on the PTP clock module are cached in the HBM memory. The video, audio, and auxiliary data in the HBM memory are extracted to the signal up / down conversion processing module, and the signal up / down conversion is completed according to the user configuration requirements provided by ARM, and the converted data is sent to the HBM memory; the signal up / down conversion includes: signal frame synchronization, 8K / 4K image resolution conversion, 8K / 4K video brightness curve, WCG color gamut conversion, and 8K / 4K signal frame rate conversion; The converted data in the HBM memory is sent to the ST2110 protocol packet module, and the uncompressed ST2110 protocol IP data packet is completed based on the PTP clock module. Then the packetized data is sent to the network protocol packet module for network multiplexing, and finally the processed IP stream is sent back to the external core switch via the 100G optical fiber network interface; The signal control module provides the device with an external interface communication protocol through a protocol stack, and provides a client web configuration interface for users to modify working parameters and view working status.
4. The 8K / 4K ultra-high-definition native IP media signal format conversion method according to claim 3, characterized in that: In the HBM memory, a method for extending the read and write bandwidth of the on-chip cache is used to implement writing and reading of ultra-high bandwidth data, specifically including: When controlling access, a timing control signal for HBM is generated based on the FPGA clock; Assign values to the AXI-ports. The corresponding address channels and HBM channels under each AXI-port have corresponding addresses to construct a storage matrix containing M AXI-ports. When data is written, the data is evenly divided and written into the storage matrix from M AXI-ports; when data needs to be read, it is read from multiple AXI-ports at the same time.
5. The 8K / 4K ultra-high-definition native IP media signal format conversion method according to claim 4, characterized in that: When writing data to HBM through AXI-port, a new address algorithm with the same address bits and different data bits is used. The new address consists of the particle ID and the particle access address. The algorithm controls the writing and reading of data to the new address.
Citation Information
Cited By
Embedded system architecture of NMOS (N-channel Metal Oxide Semiconductor) service, construction method and equipment
CN120512564A
