Extensible multi-path image storage and attitude analysis cooperation system

Abstract

The invention discloses an extensible multi-path image storage and attitude analysis collaborative system, which relates to the technical field of image data processing, has strong storage capacity expansibility, can linearly expand the storage capacity by increasing the number of hard disks, and meets the long-term data storage requirement. The image splicing real-time performance is obviously improved, the hardware-level splicing design is adopted, and the splicing delay is smaller than 10 us. Characteristic calculation efficiency is broken through in parallel, a heterogeneous architecture (FPGA + mercuric chloride 310B calculation) is combined with a PC I e high-speed link, and the calculation power utilization rate is increased by 300% compared with a single chip. Module coordination is high, calculation precision is high, an FPGA device in the feature processing module can flexibly allocate tasks according to the resource utilization rate of the mercuric chloride 310, and dynamic adjustment of the bandwidth of a PC I e link is achieved. By adopting the algorithm model optimized by the CANN architecture, the calculation efficiency is improved by 40%, and the error rate of a calculation result is less than 0.5%.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a scalable multi-channel image storage and attitude resolution collaborative system based on ZYNQ and Ascend 310. Background Technology

[0002] Image data processing technology includes multi-channel image storage and stitching technology, large-capacity anti-interference storage, and dynamic expansion of heterogeneous computing power.

[0003] Existing image processing techniques have the following drawbacks:

[0004] 1. Insufficient real-time performance: Multi-channel image storage relies on CPU or general-purpose GPU software algorithms to achieve timestamp alignment and frame merging of multi-channel images, with a latency of ≥100ms.

[0005] 2. Low storage reliability: Single-path fiber transmission is susceptible to interference and has a high data packet loss rate.

[0006] 3. Inflexible computing power expansion: The highest version of the single Ascend 310B chip has a computing power of only 20T, which is insufficient to meet the demand for high computing power.

[0007] 4. Insufficient accuracy in attitude resolution: The general YOLO model is not optimized for target features, resulting in low computational efficiency and large errors in attitude angle calculation. Summary of the Invention

[0008] In view of the above problems, the present invention provides a scalable multi-channel image storage and pose resolution collaborative system for overcoming or at least partially solving the above problems.

[0009] This invention provides the following solution:

[0010] A scalable multi-channel image storage and pose resolution collaborative system includes:

[0011] The data storage module includes a Z7 ZYNQ chip, and the PL terminal on the Z7 ZYNQ chip includes a DTRU module. The data storage module realizes real-time reception of target source image data through two optical fibers connected to the Z7 ZYNQ chip. The DTRU module is used to complete the stitching of the target source image data to obtain stitched image data, and add a timestamp to the header of each frame of stitched image data.

[0012] The feature processing module includes an FPGA device and an Ascend 310 chip. The FPGA device receives the stitched image data transmitted from the data storage module via a single optical fiber and preprocesses the stitched image data so that the preprocessed stitched image data is transmitted to the Ascend 310 chip via a PCIe link at a target frame rate per second. The Ascend 310 chip uses a trained inference model to process the preprocessed stitched image data to obtain calculation results and returns the calculation results to the FPGA device. The FPGA device then combines and interpolates the calculation results according to the order in which they are received and reports the calculation results to the host computer via a 422 interface at a target rate.

[0013] Preferably, the data storage module further includes a ZYNQ+16T radiation-resistant NVMe SSD storage array.

[0014] Preferably, the stitched image data is preprocessed and stored in a cache. After the task is started, the preprocessed stitched image data is transmitted to the Ascend 310 chip via a PCIe link at a rate of target frames per second.

[0015] Preferably, the preprocessing includes target region extraction, noise reduction, and background reduction.

[0016] Preferably, a high-speed serial port is provided between the Ascend 310 chip and the FPGA device, so that the FPGA device can obtain the resource utilization rate of the Ascend 310 chip and dynamically allocate bandwidth according to the resource utilization rate through the built-in PCIe controller.

[0017] Preferably, the Ascend 310 chip comprises three chips.

[0018] Preferably, the inference model includes an attitude analysis model.

[0019] Preferably, the attitude analysis model is trained based on the characteristics of the target being attacked, and adopts a quantized network structure and quantization acceleration technology.

[0020] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0021] This application provides a scalable multi-channel image storage and pose analysis collaborative system. It boasts strong storage capacity scalability, linearly expanding storage capacity by increasing the number of hard drives to meet long-term data storage needs. Image stitching real-time performance is significantly improved, with hardware-level stitching design achieving a stitching latency of less than 10µs. Feature computation efficiency is significantly improved through parallel processing, utilizing a heterogeneous architecture (FPGA + Ascend 310B computing) combined with a high-speed PCIe link, increasing computing power utilization by 300% compared to a single chip. The system exhibits strong module coordination and high computational accuracy; the FPGA device in the feature processing module can flexibly allocate tasks based on the Ascend 310 resource utilization, enabling dynamic adjustment of PCIe link bandwidth. The algorithm model optimized using the CANN architecture improves computational efficiency by 40%, with a computational error rate of less than 0.5%.

[0022] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0024] Figure 1 This is a schematic diagram of a scalable multi-channel image storage and pose resolution collaborative system provided in an embodiment of the present invention;

[0025] Figure 2 This is an overall block diagram of the data storage module provided in an embodiment of the present invention;

[0026] Figure 3 This is a block diagram of the PL end provided in an embodiment of the present invention;

[0027] Figure 4 This is a block diagram of the feature processing module provided in an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0029] See Figure 1 This invention provides a scalable multi-channel image storage and pose resolution collaborative system, such as... Figure 1 As shown, the system may include:

[0030] The data storage module includes a Z7 ZYNQ chip, and the PL terminal on the Z7 ZYNQ chip includes a DTRU module. The data storage module realizes real-time reception of target source image data through two optical fibers connected to the Z7 ZYNQ chip. The DTRU module is used to complete the stitching of the target source image data to obtain stitched image data, and adds a timestamp to the header of each stitched image data frame. In a specific implementation, the data storage module may also include a ZYNQ+16T radiation-resistant NVMe SSD storage array.

[0031] The feature processing module includes an FPGA device and an Ascend 310 chip. The FPGA device receives the stitched image data transmitted from the data storage module via a single optical fiber and preprocesses the stitched image data so that the preprocessed stitched image data is transmitted to the Ascend 310 chip via a PCIe link at a target frame rate per second. The Ascend 310 chip uses a trained inference model to process the preprocessed stitched image data to obtain calculation results and returns the calculation results to the FPGA device. The FPGA device then combines and interpolates the calculation results according to the order in which they are received and reports the calculation results to the host computer via a 422 interface at a target rate.

[0032] In a specific implementation, this application embodiment can provide and preprocess the stitched image data and store it in a cache area. After the task is determined to start, the preprocessed stitched image data is transmitted to the Ascend 310 chip through the PCIe link at the rate of target frames per second.

[0033] The preprocessing includes target region extraction, noise reduction, and background reduction.

[0034] A high-speed serial port is provided between the Ascend 310 chip and the FPGA device, so that the FPGA device can obtain the resource utilization rate of the Ascend 310 chip and dynamically allocate bandwidth according to the resource utilization rate through the built-in PCIe controller.

[0035] The Ascend 310 chip consists of three chips.

[0036] The inference model includes an attitude parsing model.

[0037] The attitude analysis model is trained based on the characteristics of the target being attacked, and employs a quantized network structure and quantization acceleration technology.

[0038] The scalable multi-channel image storage and attitude resolution collaborative system provided in this application adopts a hardware-level multi-channel image stitching mechanism. It utilizes the hardware parallelism of ZYNQ's PL end to achieve timestamp alignment (accuracy 1us) and frame merging of two fiber inputs, merging multiple images into a single continuous stream according to timestamps, with a stitching delay of less than 10us (99% lower than the software solution).

[0039] The scalable multi-channel image storage and attitude analysis collaborative system provided in this application adopts a radiation-resistant large-capacity storage architecture and uses a ZYNQ+16T radiation-resistant NVMe SSD array. It combines two 10G fiber inputs into a single 10G fiber output through hardware-level data forwarding, reducing storage bandwidth requirements by 50%. It supports RAID 5 redundancy and ECC error correction coding, and its resistance to space radiation is improved by three orders of magnitude compared to ordinary storage.

[0040] The scalable multi-channel image storage and attitude analysis collaborative system provided in this application adopts a dynamic PCIe multi-chip computing power pool, supporting up to three PCIe channels connected in parallel to three Ascend 310B chips. Dynamic bandwidth allocation is achieved through the PCIe controller built into the FPGA (adjusting the single-channel PCIe transmission rate according to the workload); the computational throughput can theoretically be increased linearly by up to three times.

[0041] The scalable multi-channel image storage and attitude analysis collaborative system provided in this application adopts a customized attitude analysis model for the strike target. The attitude analysis model loaded on the Ascend 310B is trained based on the characteristics of the strike target (low texture, small target, low illumination). It adopts a quantized network structure and quantization acceleration technology (FP16→INT8), which improves the computational efficiency by 40% and the error rate of the calculation result is less than 0.5%.

[0042] The system provided in this application will be described in detail below.

[0043] like Figure 1 As shown, the system is mainly divided into two parts: a feature processing module and a data storage module. The data storage module is responsible for receiving, stitching, and forwarding multi-channel fiber optic image data, and for storing and reporting the image data. The feature processing module mainly issues commands to the storage module, performs calculations and parsing on the target image, and reports the calculation results.

[0044] like Figure 2 , Figure 3The diagram shows the overall block diagram and PL (Plug and Display) terminal block diagram of the data storage module. The data storage module is mainly constructed from the Z7ZYNQ chip and peripheral circuitry. The data storage module uses two optical fibers to receive and stitch target source image data in real time. These two fibers are connected to the PL terminal of the ZYNQ chip. Stitching is performed independently of traditional software solutions; the DTRU module on the PL terminal performs the image stitching, with a stitching latency of less than 10µs. Simultaneously, a timestamp is added to the header of each stitched image frame to ensure traceability. The stitched image data is then output to the feature processing module via a single optical fiber. The data storage module is equipped with a large-capacity radiation-resistant NVMe SSD storage array (RAID5+ECC) to improve the reliability of the storage module and reduce the bit error rate. It can also perform operations such as image data storage, reporting, and frame extraction according to the work instructions issued by the feature processing module.

[0045] like Figure 4 As shown, the feature processing module mainly consists of FPGA devices, Ascend 310 chips, and peripheral circuits. First, the FPGA device in the feature processing module receives image data transmitted from the storage module via a single optical fiber and preprocesses the image data (target region extraction, noise reduction, and background reduction). During the task preparation phase, the preprocessed image data is placed in the buffer (DDR3). Then, at the start of the task, the image data is transmitted to the three Ascend 310 chips via PCIe links at a default speed of 500 frames per second (adjustable). A high-speed serial port is provided between the Ascend 310 and the FPGA device to allow the FPGA to obtain the resource utilization rate of the Ascend 310. Based on the resource utilization rate, the FPGA device dynamically allocates bandwidth through its built-in PCIe controller. The Ascend 310 returns the calculation results of the inference model (which has been pre-trained, optimized, and deployed using a low-texture, small-object, low-light data source for model training, and optimized via Huawei CANN heterogeneous algorithm platform FP16→INT8 and deployed) to the FPGA device through the PCIe link. The FPGA device combines and interpolates the calculation results according to the order in which they are received, and reports the calculation results to the host computer at a fixed rate of 200Hz through the 422 interface.

[0046] In summary, the scalable multi-channel image storage and pose analysis collaborative system provided in this application offers strong storage capacity scalability, which can be linearly expanded by increasing the number of hard drives to meet long-term data storage needs. Image stitching real-time performance is significantly improved, with hardware-level stitching design achieving a stitching latency of less than 10µs. Parallel breakthroughs in feature computation efficiency are achieved through a heterogeneous architecture (FPGA + Ascend 310B computing) combined with a high-speed PCIe link, increasing computing power utilization by 300% compared to a single chip. Strong module coordination and high computational accuracy are achieved; the FPGA devices in the feature processing module can flexibly allocate tasks based on the Ascend 310 resource utilization, enabling dynamic adjustment of PCIe link bandwidth. The algorithm model optimized using the CANN architecture improves computational efficiency by 40%, with a computational error rate of less than 0.5%.

[0047] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0048] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0049] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A scalable multi-channel image storage and pose resolution collaborative system, characterized in that, include: The data storage module includes a Z7 ZYNQ chip, and the PL terminal on the Z7 ZYNQ chip includes a DTRU module. The data storage module realizes real-time reception of target source image data through two optical fibers connected to the Z7 ZYNQ chip. The DTRU module is used to complete the stitching of the target source image data to obtain stitched image data, and add a timestamp to the header of each frame of stitched image data. The feature processing module includes an FPGA device and an Ascend 310 chip. The FPGA device receives the stitched image data transmitted from the data storage module via a single optical fiber and preprocesses the stitched image data so that the preprocessed stitched image data is transmitted to the Ascend 310 chip via a PCIe link at a target frame rate per second. The Ascend 310 chip uses a trained inference model to process the preprocessed stitched image data to obtain calculation results and returns the calculation results to the FPGA device. The FPGA device then combines and interpolates the calculation results according to the order in which they are received and reports the calculation results to the host computer via a 422 interface at a target rate.

2. The scalable multi-channel image storage and pose resolution collaborative system according to claim 1, characterized in that, The data storage module also includes a ZYNQ+16T radiation-resistant NVMe SSD storage array.

3. The scalable multi-channel image storage and pose resolution collaborative system according to claim 1, characterized in that, After preprocessing the stitched image data, it is stored in the cache. After the task is determined to start, the preprocessed stitched image data is transmitted to the Ascend 310 chip via the PCIe link at the rate of target frames per second.

4. The scalable multi-channel image storage and pose resolution collaborative system according to claim 1, characterized in that, The preprocessing includes target region extraction, noise reduction, and background reduction.

5. The scalable multi-channel image storage and pose resolution collaborative system according to claim 1, characterized in that, A high-speed serial port is provided between the Ascend 310 chip and the FPGA device, so that the FPGA device can obtain the resource utilization rate of the Ascend 310 chip and dynamically allocate bandwidth according to the resource utilization rate through the built-in PCIe controller.

6. The scalable multi-channel image storage and pose resolution collaborative system according to claim 1, characterized in that, The Ascend 310 chip consists of three chips.

7. The scalable multi-channel image storage and pose resolution collaborative system according to claim 1, characterized in that, The inference model includes an attitude parsing model.

8. The scalable multi-channel image storage and pose resolution collaborative system according to claim 7, characterized in that, The attitude analysis model is trained based on the characteristics of the target being attacked, and employs a quantized network structure and quantization acceleration technology.

Images