Ultrasound Mp soc + +-based high-speed camera image acquisition device and method

Abstract

The invention discloses a high-speed camera image acquisition device and method based on Ultrasound Mp soc + +. The high-speed camera image acquisition device based on Ultrasound Mp soc + + comprises a camera, a Zynq Ultrasound Scale + MPSoC chip, two DDR3SDRAMs, an SFP + optical module, a PS end network port and an upper computer. Wherein the camera sends generated pixel data to the SFP + optical module through the GTH interface, the SFP optical port converts an optical signal into an electric signal and sends the electric signal to the DDR3SDRAM at the PL end, the DDR3SDRAM receives the pixel data, image enhancement processing is carried out on the pixel data by utilizing the high-performance parallel processing capability of the PL end of the Zynq UltraScale + MPSoC chip, and a frame number, time information and a check bit are added; and forming frame-by-frame image data. Image data are stored in a fifo module for caching, then the processed image data are stored in a second DDR3SDRAM on one side of a PS end at a PL end through an AXI bus, an application program of the PS end controls to copy the image data stored in the second DDR3SDRAM and sends the data through a network port, and an upper computer receives the data through the network port, verifies the data and then displays the image in real time; according to the invention, image data can be acquired at a high speed and displayed in real time, and the problems of low real-time response speed and waste of storage resources and computing resources of an existing image data acquisition system are solved.

Description

Technical Field

[0001] This invention relates to the field of recording or reproducing devices for sound or images, specifically to a high-speed camera image acquisition device and method based on Ultrascal MP soc++. Background Technology

[0002] With societal progress and technological advancements, the demand for image acquisition and enhancement systems in key areas such as security monitoring, traffic management, and industrial automation is showing a significant upward trend. However, existing image acquisition systems often face problems such as low frame rates, inappropriate data processing, and low storage speeds.

[0003] First, the low frame rate limits the system's real-time performance and response speed. A low frame rate may result in less smooth capture of dynamic scenes, which could affect the accuracy of decision-making in applications that need to accurately record rapidly changing situations.

[0004] Secondly, insufficient data processing capabilities are also a pressing issue. Modern image acquisition equipment generates massive amounts of data, especially in the case of high-resolution and high-speed shooting. How to efficiently process this large amount of data, extract useful information, and perform real-time analysis is a major challenge for current image acquisition and enhancement systems. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides an image acquisition and enhancement system based on Zynq UltraScale+ MPSoC, which improves data transmission rate and solves the problems of slow real-time response and wasted storage and computing resources in existing image acquisition systems.

[0006] This invention solves the aforementioned technical problems through the following technical means: an image acquisition system based on Zynq UltraScale+ MPSoC, comprising a camera, a Zynq UltraScale+ MPSoC chip, two DDR3 SDRAMs, an SFP+ optical module, a PS-end network port, and a host computer. The Zynq chip includes a PL (programmable logic unit) and a PS (processing system unit). The PL includes a pixel data conversion module, a pixel data processing module, a frame information addition module, an image data enhancement module, a Data_to_AXI_Full module, an AXI_Interconnect_1 module, a Data_to_AXI_Lite module, an AXI_Interconnect_2 module, a DDR3 control module, and an Aurora module. The PS includes a network port driver and an application program.

[0007] The camera driver module is used to automatically detect and acquire all the functional characteristics supported by the camera. This step ensures that the system can fully utilize the camera's capabilities and provides accurate basic information for subsequent configuration. It is also used to configure camera parameters and initialize the camera, i.e., configure the camera's resolution, frame rate, pixel data bit width, pixel data format, etc. The driver module is also responsible for controlling the camera's output mode, such as selecting continuous stream mode or trigger mode to adapt to different application requirements, and determining whether the camera parameters have been set successfully.

[0008] The Aurora module at the PL end converts the received optical signal into an electrical signal, which is then transmitted to the back-end DDR3 control module via an optical fiber link. Optical fiber communication is known for its low loss, high bandwidth, and resistance to electromagnetic interference, enabling the Aurora module to achieve stable data transmission over long distances and in complex electromagnetic environments.

[0009] The DDR3 control module is used to cache the received image data to prevent problems such as pixel data loss caused by excessively high frame rates and large data volumes.

[0010] The pixel data conversion module at the PL end is responsible for performing a series of conversion processes on the raw pixel data received from the camera. Specifically, this module first parses the timing sequence of the data interface output by the camera and reassembles the received pixel data into a continuous pixel data stream. Subsequently, it further processes this data stream, formatting it into a YCbCr format pixel data stream for compression processing by the subsequent pixel data processing module.

[0011] The pixel data processing module at the PL end is specifically designed to execute efficient image compression algorithms. This module can compress raw image data into JPEG format in real time, significantly reducing the amount of data transmitted to the host computer, thereby effectively reducing network bandwidth requirements and storage system pressure. By integrating advanced compression technology, this module not only ensures image quality but also optimizes data flow management, enabling the system to significantly reduce unnecessary resource consumption while maintaining high performance. Ultimately, the compressed image data can be transmitted more smoothly and is easier to store and process on the host computer.

[0012] The frame information addition module at the PL end is used to add a frame number and a time tag to each frame of pixel data to form a frame of image data, and output the image data with added frame number and time tag to the image enhancement processing module.

[0013] The image enhancement processing module possesses powerful parallel processing capabilities, enabling simultaneous operations on multiple pixels and efficiently performing pixel-by-pixel processing of the entire image. It enhances low-light images of objects by deploying deep learning image enhancement algorithms and attention mechanisms. This module not only supports basic functions such as brightness adjustment and contrast enhancement but also integrates advanced noise reduction and detail enhancement techniques, significantly improving image quality and clarity.

[0014] The Data_to_AXI_Full module at the PL end is used to convert image data into AXI_Full image data and send the converted AXI_Full image data to the AXI_Interconnect_1 module. It is also used to output a pulse signal to the Data_to_AXI_Lite module after sending a burst of image data.

[0015] The image data transmission completion interrupt module Data_to_AXI_Lite at the PL end is used to receive pulse signals, convert the pulse signals into high-level signals, store them in a register, and output them to the PS end as an interrupt signal. When the PS end detects the interrupt signal, it copies the burst image data from the Data_to_AXI_Full module to the buffer and clears the register through the AXI_Interconnect_1 module.

[0016] The AXI_Interconnect_1 module on the PL side is used to receive image data of the AXI_Full protocol and output the image data to the PS side, and then store it in the DDR3 SDRAM of the PS side. It is also used to receive instruction data from the PS side and output it to the image data transmission completion interrupt module to clear the register.

[0017] The AXI_Interconnect_2 module on the PL side is used to receive the interrupt pulse of the AXI_Lite protocol and output the image data to the DDR3 control module. When the PS side detects that the pulse is high, it copies the burst image data from the Data_to_AXI_Full module to the buffer for transmission to the host computer via the network port.

[0018] The application program on the PS side is used to copy the image data sent from the PL side to the PS side DDR3 via the AXI protocol to the buffer. After detecting the pulse emitted by the Data_to_AXI_Lite module, it uploads the buffer data to the PC host computer via the network port.

[0019] After being driven and configured by the PS terminal program, the network port of the PS terminal is responsible for uploading the image data in the buffer to the host computer. The PS terminal can not only manage the transmission process of image data, but also optimize the transmission efficiency and ensure data integrity.

[0020] The PC-based host computer is responsible for receiving and decoding image data, verifying and detecting errors, processing and displaying image data from front-end devices (such as cameras or FPGA modules), and provides a series of advanced functions to enhance user experience and data management such as data storage.

[0021] The beneficial effects of this invention are as follows: By adding a pixel data processing module to the PL end of the Zynq UltraScale+ MPSoC chip, adaptive histogram equalization is performed on the pixel data to enhance image clarity and contrast, reducing image distortion caused by environmental factors and improving the imaging effect in this image acquisition system and on the backend or terminal server. Furthermore, this invention introduces the JPEG compression algorithm and deploys it on the Zynq UltraScale+ MPSoC chip to achieve color space conversion, sampling, quantization, and encoding of image data. This achieves efficient image compression of the original image data stream, significantly reducing the data transmission and storage requirements, lowering the consumption of storage and computing resources, improving the overall system operating efficiency and response speed, and reducing the data receiving pressure on the host computer. In addition, image enhancement operations are implemented, increasing the brightness and contrast of the acquired image. The Zynq UltraScale+ MPSoC used in this invention integrates a multiprocessor system (PS end) and programmable logic (PL end), achieving tight coupling and efficient collaboration between hardware and software. This heterogeneous computing platform enables the system to complete the entire process from pixel data reception, processing, detection, and transmission on a single chip, reducing overall system complexity and improving processing speed and energy efficiency. The invention employs a dual DDR3 SDRAM storage strategy in its system design. Two DDR3 SDRAM chips serve the PL (Plug-in) and PS (Power-on) ends respectively. The first DDR3 SDRAM chip on the PL end stores image data emitted by the CMOS camera, while the second DDR3 SDRAM chip on the PS end stores image data transmitted to the PS end. This design ensures high efficiency and reliability in data transmission. After detecting the completion flag pulse of a single frame of image data burst, the data in the PS end's DDR chip can be immediately copied to a buffer and uploaded to the host computer via the network port for real-time display, ensuring that complete image data can be quickly displayed and saved. Attached Figure Description

[0022] Figure 1 This invention provides an image acquisition system based on Zynq UltraScale+ MPSoC. Figure 2 Working block diagram of Zynq UltraScale+ MPSoC chip Detailed Implementation The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application.

[0024] One embodiment provides an image acquisition and enhancement system based on Zynq UltraScale+ MPSoC. For example... Figure 1As shown, this embodiment of a high-speed camera image acquisition system based on a Zynq UltraScale+ MPSoC includes a camera, a Zynq UltraScale+ MPSoC chip, two DDR3 SDRAMs, and an SFP+ optical module. The camera sends the generated pixel data to the Aurora module at the PL end, which converts the received optical signal into an electrical signal and transmits it via optical fiber to the back-end DDR3 control module. The DDR3 SDRAM at the PL end receives the pixel data and utilizes the high-performance parallel processing capabilities of the PL end to perform pixel conversion operations on the pixel data. Image compression is also performed to reduce the load on the host computer and improve storage resource utilization, while adding frame number and time information to form frame-by-frame image data. Then, the image data is stored in the Data_to_AXI_Full module on the PL side. This module converts the image data into AXI_Full image data and sends the converted AXI_Full image data to the AXI_Interconnect_1 module. It also outputs a pulse signal to the Data_to_AXI_Lite module after sending a burst of image data. The Data_to_AXI_Lite module receives the pulse signal, converts it into a high-level signal, registers it in a register, and outputs it to the PS side as an interrupt signal. When the PS side detects this interrupt signal... The image data burst by the Data_to_AXI_Full module is copied to the buffer and the register is cleared by the AXI_Interconnect_1 module. The image data is output to the PS end through the AXI_Interconnect_1 module on the PL end, and then stored in the DDR3 SDRAM on the PS end. The interrupt data is output to the PS end through the AXI_Interconnect_2 module on the PL end. When the PS end detects that this pulse is high, it copies the image data burst by the Data_to_AXI_Full module to the buffer for transmission to the host computer via the network port. The host computer stores the data and displays the image in real time.

[0025] like Figure 2The diagram shown is a block diagram of the Zynq UltraScale+ MPSoC chip. The Zynq UltraScale+ MPSoC chip includes a PL (Programmable Logic Unit) and a PS (Processing System Unit). The PL includes a pixel data conversion module, a pixel data processing module, a frame information addition module, an image data preprocessing module, a Data_to_AXI_Full module, an AXI_Interconnect_1 module, a Data_to_AXI_Lite module, an AXI_Interconnect_2 module, a DDR3 control module, and an Aurora module. The PS includes a network port driver and an application program, and the PC includes a host computer.

[0026] The camera driver on the PL side is used to read camera parameters to obtain the functions that the camera can support, configure camera parameters, initialize the camera, that is, configure the camera resolution, frame rate, pixel data bit width, pixel data format, etc., and determine whether the camera parameters are set successfully.

[0027] The pixel data conversion module on the PL side receives pixel data from the DDR3 output data interface, converts the pixel data stream into an RGB888 format pixel data stream, and outputs it to the pixel data processing module.

[0028] The pixel data processing module on the PL side compresses pixel data using an improved JPEG algorithm, effectively reducing network bandwidth requirements and storage system pressure. Integrating advanced compression technology not only ensures image quality but also optimizes data stream management, thereby ensuring the system maintains high performance while significantly reducing unnecessary resource consumption.

[0029] The frame information addition module on the PL end adds a frame number and time tag to each frame of pixel data to form a frame of image data, and outputs the image data with added frame number and time tag to the Data_to_AXI_Full module.

[0030] The image enhancement module on the PL side processes data from multiple pixels simultaneously, performing pixel-by-pixel operations on the entire image, such as brightness adjustment and contrast enhancement. It uses adaptive filters or non-local mean filtering techniques to remove random noise and improve image clarity. It also expands the grayscale range through linear or non-linear transformations to make image details more apparent. The processed image data is then transferred to the Data_to_AXI_Full module.

[0031] The Data_to_AXI_Full module on the PL side converts the image data into AXI_Full image data and sends the converted AXI_Full image data to the AXI_Interconnect_1 module. In addition, after sending one frame of image data, it outputs a pulse signal to the image data transmission completion interrupt module.

[0032] The image data transmission interrupt module at the PL end can receive pulse signals, convert them into high-level signals, store them in a register, and output them to the PS end as an interrupt signal. When the PS end detects the interrupt signal, it will clear the register through the AXI_Interconnect_1 module.

[0033] The AXI_Interconnect_1 module on the PL side is used to receive image data of the AXI_Full protocol and output the image data to the S_AXI_HP interface on the PS side, and then store it in the second DDR3 SDRAM on the PS side. It is also used to receive instruction data from the PS side and output it to the image data transmission completion interrupt module to clear the register.

[0034] The Data_to_AXI_Lite module on the PL side converts the interrupt data into AXI_Lite image data and sends the converted AXI_Lite image data to the AXI_Interconnect_2 module.

[0035] The AXI_Interconnect_2 module on the PL side receives image data from the AXI_Lite protocol and outputs the interrupt data to the DDR3 control module on the PS side through the S_AXI_GP interface. In addition, after receiving the interrupt signal, the PS side copies the image data of that frame to the buffer, and the PS side program sends the buffer data to the host computer through the Ethernet port.

[0036] The Ethernet driver on the PS side is used to initialize the network port, that is, to configure various parameters for interrupt and packet transmission error detection. It is responsible for constructing and parsing Ethernet frames, encapsulating data from the upper-layer protocol stack into a format suitable for network transmission, and sending it out through the physical layer. Similarly, it also unpacks received data packets, extracts the payload and passes it to the upper-layer application, and sends the data packets to the PC host computer.

[0037] The application on the PS side is used to handle the completion interrupt and buffer copy operation of sending a frame of image data on the PL side. After receiving the interrupt signal, it reads the frame of image data from the second DDR3 SDRAM, performs a series of post-processing operations, and copies the final image data to the buffer.

[0038] The PC-based host computer receives image data from the network port and verifies it. If the verification fails, the image frame is not displayed, the frame information is removed, the image data is saved, and the image is displayed in real time on the window.

[0039] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0040] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A high-speed camera image acquisition device based on Ultrascal MP SOC++, characterized in that, The system includes a camera, a Zynq UltraScale+ MPSoC chip, two DDR3 SDRAMs, an SFP+ optical module, a PS-side network port, and a host computer. The Zynq chip includes a PL (Plug-in) and a PS (Power Supply) side. The PL side includes a pixel data conversion module, a pixel data processing module, a frame information addition module, an image data enhancement module, a Data_to_AXI_Full module, an AXI_Interconnect_1 module, a Data_to_AXI_Lite module, an AXI_Interconnect_2 module, a DDR3 control module, an Aurora module, the PS side, a network port driver, and an application program. The camera configuration interface (SCCB) is connected to the PS side of the Zynq UltraScale+ MPSoC chip, the camera pixel data interface is connected to the PL side of the Zynq UltraScale+ MPSoC chip, the first DDR3 SDRAM is connected to the PL side of the Zynq UltraScale+ MPSoC chip, and the second DDR3 SDRAM is connected to the PS side of the Zynq UltraScale+ MPSoC chip. Image data containing the target is transmitted through the Zynq UltraScale+... The MPSoC chip's GTH interface outputs to the SFP+ optical module, which converts the electrical signal into an optical signal and transmits it to the back-end or terminal server via optical fiber.

2. A high-speed camera image acquisition method based on Ultrascal MP SOC++, characterized in that, Includes the following steps: Step 1: The camera sends the generated pixel data to the Aurora module on the PL end, which converts the received light signal into an electrical signal and sends it to the back-end DDR3 control module through optical fiber. The PL end DDR3 SDRAM receives the pixel data and uses the high-performance parallel processing capability of the PL end to perform pixel conversion operation on the pixel data, compress the image data to reduce the pressure on the host computer and the utilization of storage resources, and add frame number and time information to form frame by frame of image data. Step 2: The Zynq UltraScale+ MPSoC chip then stores the image data into the Data_to_AXI_Full module on the PL side. This module converts the image data into AXI_Full image data and sends the converted AXI_Full image data to the AXI_Interconnect_1 module. It also outputs a pulse signal to the Data_to_AXI_Lite module after sending a burst of image data. The Data_to_AXI_Lite module receives the pulse signal, converts it into a high-level signal, registers it in a register, and outputs it to the PS side as an interrupt signal. When the PS side detects the interrupt signal, it copies the burst image data from the Data_to_AXI_Full module to the buffer and clears the register through the AXI_Interconnect_1 module. The image data is then output to the PS side through the AXI_Interconnect_1 module on the PL side and stored in the PS side's DDR3 SDRAM. Step 3: The application on the PS side of the Zynq UltraScale+ MPSoC chip outputs the interrupt data to the PS side through the AXI_Interconnect_2 module on the PL side. When the PS side detects that this pulse is high, it copies the burst image data from the Data_to_AXI_Full module to the buffer for transmission to the host computer via the network port. The host computer stores the data and displays the image in real time.

3. The high-speed camera image acquisition method based on Ultrascal MP soc++ according to claim 2, characterized in that: In step 1, the Zynq UltraScale+ MPSoC chip uses high-performance parallel capabilities at the PL end to perform image enhancement operations on pixel data through deep learning algorithms and attention mechanisms, and then adds frame number and time information.

4. The high-speed camera image acquisition method based on Ultrascal MP SOC++ according to claim 1, characterized in that: In step 2, the Zynq UltraScale+ MPSoC chip utilizes its high-performance parallel capabilities at the PL end to perform image data transmission operations via the AXI4 protocol, reducing the time consumed by subsequent image transmission.

5. The high-speed camera image acquisition method based on Ultrascal MP SOC++ according to claim 1, characterized in that: In step 2, after the Zynq UltraScale+ MPSoC chip's PL side sends a frame of image data after the resize operation to the second DDR3 SDRAM on the PS side, the Data_to_AXI_Full_1 module on the PL side outputs a pulse signal to the image data transmission completion interrupt module. After receiving the pulse signal, the image data transmission completion interrupt module converts the pulse signal into a high-level signal, stores it in the register, and outputs it to the PS side as an interrupt signal. When the PS side detects the interrupt signal, it will transmit the frame of image data to the host computer, and at the same time clear the register through the AXI_Interconnect_1 module.

6. The high-speed camera image acquisition method based on Ultrascal MP SOC++ according to claim 1, characterized in that: In step 2, two DDR3 SDRAM chips are used to store the image data to be sent to the backend or terminal server and the image data to be transmitted, respectively, ensuring the high efficiency of data transmission.

7. The high-speed camera image acquisition method based on Ultrascal MP SOC++ according to claim 1, characterized in that: In step 3, the PS side of the Zynq UltraScale+ MPSoC chip sends the post-processed image inference information containing the target to the interrupt register module of the PL side via the AXI bus. When the image data containing the target is sent to the backend or terminal server, the image inference information stored in the interrupt register module will be cleared.

Images