High-speed snapshot system and method

Abstract

The invention discloses a high-speed snapshot system and method, and relates to the technical field of image acquisition. The system comprises a trigger detection module, an image acquisition module, a processing and control module, a DDR cache chip and a storage and transmission module. The image acquisition module adopts a high-speed CMOS (Complementary Metal-Oxide-Semiconductor Transistor) image sensor and works in a high-frame-rate mode which is not lower than 120 frames per second under a preset resolution ratio; the processing and control module adopts a system-level chip integrated with an image signal processor and a hardware encoder; the DDR cache chip is connected with the processing and control module through a memory bus and serves as a high-speed data buffer pool. Through cooperative work of a high-frame-rate sensor and a hardware-level encoder and cooperation of independent DDR cache, clear, continuous and frame-loss-free snapshot of high-speed moving targets such as flying insects is realized, and the problems of image blurring and incomplete data caused by insufficient frame rate and limited data processing capability in the prior art are solved. The system is high in integration level, flexible in deployment and suitable for field monitoring and laboratory research.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image acquisition and processing, in particular to a high-speed snapshot system and method. BACKGROUND

[0002] In many important fields such as pest control, disease transmission monitoring and insect behavior research, it is of great significance to obtain high-speed motion images of flying insects (such as mosquitoes) in the air. Such image data not only helps accurate identification of species and individual counting, but also provides a basis for in-depth research such as wing vibration frequency analysis and motion trajectory reconstruction. However, due to the small size, fast movement and high wing vibration frequency of flying insects, traditional imaging systems face serious challenges in frame rate, exposure time, trigger response speed and data processing capacity, and it is often difficult to achieve clear, continuous and complete capture of their motion process.

[0003] In the prior art, some solutions aimed at improving the timeliness of snapshot have appeared. For example, a snapshot control device and method are disclosed in Chinese Patent No. CN116916142B, which wakes up the camera in advance before the target enters the center area of the camera by setting the cooperative work of the side sensor and the center sensor, so as to reduce the risk of missing the shot caused by the slow cold start of the camera. This solution effectively improves the trigger response delay of the system. However, the applicant has found through in-depth research that even if the trigger delay problem is solved, for flying insects and other high-speed small targets, to achieve high-quality clear imaging and complete data recording, the prior art still has the following significant deficiencies: Firstly, in terms of imaging frame rate and image quality, the existing system usually uses an ordinary image sensor with limited frame rate. When maintaining a high resolution, its frame rate is often difficult to break through 60 frames per second, resulting in the inability to effectively capture the high-frequency vibration of insect wings and other transient postures, and the image has serious motion blur, which seriously affects the accuracy of subsequent feature extraction and analysis.

[0004] Secondly, in terms of data processing and throughput, the massive data generated by high-speed image acquisition puts high requirements on the real-time processing and storage capacity of the system. Limited by the software coding capacity of general-purpose processors and the transmission bandwidth of common interfaces, the system often faces huge data processing pressure during continuous snapshot, and is prone to coding bottlenecks, buffer overflow, even frame loss and other problems, ultimately damaging the integrity and usability of the data.

[0005] In summary, although the prior art has improved the trigger response, there are obvious deficiencies in the synergistic optimization of high frame rate acquisition, high-quality imaging and high-throughput data processing. The improvement of trigger speed is only the first step, and if the backend imaging and processing system cannot match, it is still impossible to complete the reliable snapshot of high-speed micro targets. Therefore, the industry urgently needs a high-speed snapshot system that optimizes the image sensor performance, core processing architecture, data pipeline management, etc. SUMMARY

[0006] The purpose of the present application is to provide a high-speed snapshot system and method to solve the technical problem that the low imaging frame rate and insufficient data processing capacity in the prior art cannot clearly, continuously and completely snapshot high-speed micro targets.

[0007] To achieve the above-mentioned purpose, the present application adopts the following technical solutions: In a first aspect, the present application provides a high-speed snapshot system, characterized in that it comprises a trigger detection module, an image acquisition module, a processing and control module, a DDR cache chip and a storage and transmission module; the output end of the trigger detection module is connected with the trigger input end of the processing and control module; the image acquisition module is connected with the image input interface of the processing and control module through a high-speed serial interface; the processing and control module is connected with the DDR cache chip through its memory bus; the storage system interface of the processing and control module is connected with the storage and transmission module; wherein the image acquisition module comprises a high-speed CMOS image sensor, the high-speed CMOS image sensor is configured to output image data at a frame rate of no less than 120 frames / second; the processing and control module comprises a system-on-chip, the system-on-chip has integrated therein an image signal processor for processing raw image data and a hardware encoder for compressing and encoding processed image data; the storage and transmission module comprises a storage medium interface for local storage and a network transmission interface for remote data transmission.

[0008] As a preferred, the model of the high-speed CMOS image sensor is SC132GS, and the high frame rate mode is specifically working at a frame rate of no less than 120 frames / second under a preset resolution.

[0009] As a preferred, the model of the system-on-chip is HiSilicon HI3519D.

[0010] As a preferred, the hardware encoder is an H.264 or H.265 hardware encoder.

[0011] As a preferred, the system-on-chip further has integrated therein a memory controller, the memory controller is used for read-write access control of the DDR cache chip through the memory bus.

[0012] Preferably, the storage medium interface is an SD card controller or an eMMC controller, and the network transmission interface is a gigabit Ethernet controller.

[0013] Preferably, the system further includes an illumination module, which includes an LED light group and its driving circuit. The driving circuit is connected to the general input / output interface of the processing and control module and is controlled by a synchronization signal output by the module that is synchronized with the image acquisition exposure timing.

[0014] Preferably, the trigger detection module is any one of an infrared beam sensor, a microwave radar sensor, or a video motion detection unit.

[0015] Secondly, the present invention provides a high-speed image capture method, which applies the high-speed image capture system described above, characterized by comprising the following steps: The trigger detection module generates a trigger signal, which is then transmitted to the processing and control module. The processing and control module responds to the trigger signal by sending an acquisition command to the image acquisition module; The image acquisition module acquires an image sequence at a frame rate of not less than 120 frames per second according to the acquisition command, and transmits the raw image data to the processing and control module through the high-speed serial interface; The processing and control module performs image enhancement processing on the original image data through its internal image signal processor, and compresses and encodes the processed image data into a video stream through its internal hardware encoder. The video stream is written into the DDR cache chip for buffering; The buffered video stream is transferred from the DDR cache chip to the storage and transmission module for local storage or network transmission.

[0016] Preferably, the image enhancement processing of the original image data includes noise reduction, sharpening, and gamma correction; the compression encoding is hardware encoding using the H.264 or H.265 encoding standard.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. The unification of high frame rate and high definition is achieved, and the foundation of low latency data transmission is laid: By configuring the image acquisition module to work in high frame rate mode (preferably not less than 120fps), the premise for clearly capturing high-speed moving targets is provided. Combined with low-latency high-speed serial interface (such as MIPI CSI-2) for data transmission, low latency is achieved from the physical layer and the protocol layer: its high bandwidth characteristics provide a fast channel for massive image data, fundamentally reducing transmission time; its simplified dedicated protocol avoids complex protocol conversion, significantly reducing data processing delay. The image signal processor (ISP) integrated in the system-on-chip can perform real-time noise reduction, sharpening and other enhancement processing on the original image, effectively suppressing motion blur under high-speed shooting, and ensuring the availability of image details.

[0018] 2. A high-efficiency data processing pipeline at the hardware level is constructed: Unlike existing systems that use general-purpose processors for software coding, the present invention uses a system-on-chip that integrates a dedicated hardware encoder (such as an H.264 / H.265 encoder). This hardware encoder is designed specifically for video compression algorithms and can work independently of the main processor in parallel, achieving real-time, low-power compression encoding of high frame rate image data. This architecture frees the main processor from the heavy encoding calculations and ensures that the encoding rate matches the acquisition rate, effectively avoiding system stalls or frame loss caused by insufficient software coding calculation capacity.

[0019] 3. Efficient data management, no risk of frame loss: By introducing an independent DDR cache chip as a high-speed cache pool, a dedicated data buffer is constructed between high-speed encoding and relatively low-speed storage. This design effectively resolves the instantaneous contradiction between the high-throughput data generated by the front-end high-speed acquisition and the bandwidth of the back-end storage / transmission. The high bandwidth characteristics of the DDR cache chip, combined with the direct memory access capability of the system-on-chip, ensure that the encoded stream can be continuously and smoothly stored, thereby eliminating the risk of frame loss caused by instantaneous data peaks or back-end transmission delays, and ensuring data integrity.

[0020] 4. High system integration and flexible deployment: The present invention integrates high-speed acquisition, processing, caching, storage and transmission functions through modular design and standardized interfaces, achieving system miniaturization and high reliability. The trigger detection module can be flexibly selected according to the application scenario; the storage and transmission module simultaneously supports local and non-volatile storage and remote network transmission. Combined with the optional synchronous lighting module, the entire system can flexibly adapt to the deployment needs of various complex environments such as the field and the laboratory, and has strong practicality. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is the structural block diagram of the high-speed snapshot system provided by the first embodiment of the present invention; Figure 2 is a structural block diagram of a high-speed snapshot system (including an illumination module) provided by Embodiment Two of the present application; Figure 3 is a flowchart of a high-speed snapshot method provided by the present application; DETAILED DESCRIPTION

[0022] The technical solutions of the present application will be described clearly and completely in combination with the drawings and embodiments.

[0023] Embodiment One Referring to Figure 1 , the present embodiment provides a basic configuration of a high-speed snapshot system, mainly including a trigger detection module 1, an image acquisition module 2, a processing and control module 3, a DDR cache chip 4 and a storage and transmission module 5.

[0024] Trigger detection module 1: In the present embodiment, an infrared reflection sensor can be used. The output end thereof is connected with a general input / output (GPIO) interface of the processing and control module 3, for generating a trigger signal when a target (such as a flying insect) crosses the monitoring light beam. Those skilled in the art can understand that the trigger detection module 1 can also use a microwave radar or a video movement detection scheme.

[0025] Image acquisition module 2: The core thereof is a high-speed CMOS image sensor 21. The sensor 21 is configured to work in a high frame rate mode. In a preferred embodiment, the high frame rate mode is specifically to collect at a frame rate of 120 frames / second or higher at a resolution of 1280x1080. As a specific example of the preferred mode, the model of the sensor 21 can be SC132GS. The sensor 21 is connected with an image input interface of the processing and control module 3 through a MIPI CSI-2 high-speed serial interface, to ensure that the original image data can be transmitted at high speed and low delay.

[0026] Processing and control module 3: The core thereof is a system-on-chip (SoC) 31. In a preferred embodiment, the model of the SoC 31 is HiSilicon HI3519D. The chip 31 internally integrates an image signal processor (ISP) 311, a hardware encoder 312 and a memory controller 313.

[0027] The image signal processor (ISP) 311 is used to perform real-time pipeline processing on the original image (Bayer format) data transmitted from the MIPI interface, including automatic white balance, noise reduction, sharpening and gamma correction, to improve the image quality and effectively suppress the blur caused by the high-speed movement of the target.

[0028] The hardware encoder 312 is an H.264 or H.265 dedicated hardware encoder integrated in the SoC. It receives the image data processed by the ISP 311 and performs parallel encoding calculation independently of the main processor to compress the image data into a high compression ratio video stream in real time. This hardware-level encoding scheme has much higher encoding efficiency and real-time performance than software encoding, which is the key to enabling the system to continuously process high frame rate data streams without bottlenecks.

[0029] The memory controller 313 is used to control and manage the efficient read and write access of the DDR cache chip 4 through the DDR memory bus. Its efficiency is embodied in the following cooperative working mechanism: Priority arbitration mechanism: The memory controller 313, as the bus arbitrator for accessing DDR within the SoC, is configured to respond to the continuous stream write request of the hardware encoder 312 with priority, ensuring that high-throughput image data can be cached in real time without blocking, and eliminating the loss of front-end data caused by memory access competition from the system architecture.

[0030] Burst transmission optimization: The memory controller 313 supports and is configured to use the burst transmission mode of DDR. This mode allows a complete data block to be transmitted continuously in a single memory access operation, thereby maximizing bus utilization and effective data transmission bandwidth, which is highly matched with the working characteristics of the hardware encoder 312 continuously outputting data blocks.

[0031] Read and write parallel scheduling: By block management of the DDR cache space, the memory controller 313 can coordinate the read operation of the back-end storage interface to the cache and the write operation of the front-end encoder, to a certain extent, to realize the parallelization of read and write channels, thereby smoothing the speed difference between the front and back ends and ensuring the continuity of the data stream.

[0032] DDR cache chip 4: This is a separate DDR3 or DDR4 memory chip directly connected to the DDR memory bus of the processing and control module 3. Its core function is to serve as a high-speed cache pool for encoded video streams. Under the management of the memory controller 313 of the system-level chip 31, it temporarily stores the high-throughput video stream output from the hardware encoder 312 to smooth the data rate difference between the front-end high-speed encoding and the back-end relatively low-speed storage / transmission, and fundamentally avoid frame loss.

[0033] Storage and transmission module 5: including storage medium interface 51 and network transmission interface 52.

[0034] The storage medium interface 51 is specifically an SD card controller or eMMC controller, which is used to write the buffered video stream into the local storage medium for persistent storage.

[0035] The network transmission interface 52 is specifically a gigabit Ethernet controller, used for real-time transmission of the video stream to a remote computer or a network server, facilitating remote monitoring and data collection.

[0036] Embodiment Two Referring to Figure 2 , this embodiment is based on Embodiment One, and an illumination module 6 is added.

[0037] The illumination module 6 includes a high-brightness LED lamp set and its corresponding constant-current driving circuit. The enable end of the driving circuit is connected with a general-purpose input / output (GPIO) interface of the processing and control module 3. In operation, the processing and control module 3 accurately outputs a synchronization pulse signal to the GPIO port according to the exposure timing of the image acquisition module 2, to control the LED lamp set to perform a short (microsecond level) and high-intensity flash at the moment of camera exposure. This synchronization flash technology can provide sufficient illumination for the shooting target within a very short exposure time, effectively “freezing” the high-speed motion moment, significantly improving the signal-to-noise ratio and clarity of the image, especially when the ambient light is insufficient.

[0038] Method Embodiment Referring to Figure 3 , a high-speed snapshot method based on the above system includes the following steps: S1: After system power-on initialization, enter low-power standby state, and trigger the detection module 1 to continuously monitor the predetermined area.

[0039] S2: When the target (such as a flying insect) enters the monitoring area, the trigger detection module 1 generates a trigger signal and immediately sends it to the processing and control module 3.

[0040] S3: The processing and control module 3 responds to the trigger signal and issues a collection instruction to the image acquisition module 2. If the system is configured with an illumination module 6, the processing and control module 3 synchronously generates an illumination synchronization signal strictly aligned with the exposure timing, and outputs it to the illumination module 6.

[0041] S4: The image acquisition module 2 starts to continuously collect image sequences in a high-frame-rate mode (preferably not less than 120 frames / second) according to the instruction, and streams the raw image data to the processing and control module 3 through a high-speed serial interface such as MIPI CSI-2.

[0042] S5: The processing and control module 3 calls its internal image signal processor (ISP) 311 to perform real-time image enhancement processing (including noise reduction, sharpening, gamma correction, etc.) on the raw data. Subsequently, it calls the hardware encoder 312 to efficiently compress the processed frame data into H.264 or H.265 video stream. The hardware encoding in this step is the key to ensure real-time processing of high-frame-rate data.

[0043] S6: The generated video code stream is continuously written into the DDR cache chip 4 for temporary storage and buffering under the control of the internal memory controller of the SoC. This step is the core of solving the data throughput bottleneck by using independent cache.

[0044] S7: The processing and control module 3 reads the buffered code stream from the DDR cache chip 4 through its memory controller, and delivers the code stream to the storage and transmission module 5 through its storage system interface. The storage and transmission module 5 writes the code stream into the local SD card and / or uploads it to the network through Gigabit Ethernet according to the system configuration.

[0045] S8: After a snapshot event (e.g., the end of a preset collection time or the target leaving the monitoring area) ends, the system control logic resets and returns to the standby state (S1) again, waiting for the next trigger.

[0046] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any changes or replacements within the technical scope disclosed by the present application can be easily thought of by those skilled in the art, and should be covered within the protection scope of the present application.

Claims

1. A high-speed image capture system, characterized in that, include: The system comprises a trigger detection module (1), an image acquisition module (2), a processing and control module (3), a DDR cache chip (4), and a storage and transmission module (5); the output of the trigger detection module (1) is connected to the trigger input of the processing and control module (3); the image acquisition module (2) is connected to the image input interface of the processing and control module (3) via a high-speed serial interface; the processing and control module (3) is connected to the DDR cache chip (4) via its memory bus; the storage system interface of the processing and control module (3) is connected to the storage and transmission module (5); wherein, the image acquisition module (2) includes a high-speed CMOS image sensor (21), which is configured to output image data in a high frame rate mode; The processing and control module (3) includes a system-on-a-chip (31), which integrates an image signal processor (311) for processing the original image data and a hardware encoder (312) for compressing and encoding the processed image data; the storage and transmission module (5) includes a storage medium interface (51) for local storage and a network transmission interface (52) for remote data transmission.

2. The high-speed capture system according to claim 1, characterized in that, The high-speed CMOS image sensor (21) is model SC132GS, and the high frame rate mode specifically refers to working at a frame rate of not less than 120 frames / second at a preset resolution.

3. The high-speed capture system according to claim 1, characterized in that, The system-on-a-chip (31) is the Hisilicon HI3519D chip.

4. The high-speed capture system according to claim 1, characterized in that, The hardware encoder (312) is an H.264 hardware encoder.

5. The high-speed capture system according to claim 1, characterized in that, The system-on-a-chip (31) also integrates a memory controller (313) for reading and writing access control of the DDR cache chip (4) through the memory bus.

6. The high-speed capture system according to claim 1, characterized in that, The storage medium interface (51) is an SD card controller or an eMMC controller, and the network transmission interface (52) is a gigabit Ethernet controller.

7. A high-speed image capture system according to claim 1, characterized in that, It also includes an illumination module (6), which includes an LED lamp group and its driving circuit. The driving circuit is connected to the general input / output interface of the processing and control module and is controlled by the synchronization signal output by the module, which is synchronized with the image acquisition exposure timing.

8. A high-speed capture system according to claim 1, characterized in that, The trigger detection module (1) is any one of an infrared beam-and-beam module, a microwave radar module, or a video motion detection module.

9. A high-speed image capture method, characterized in that, The method, applied to the high-speed capture system according to any one of claims 1 to 8, comprises the following steps: The trigger detection module (1) generates a trigger signal and transmits it to the processing and control module (3). The processing and control module (3) responds to the trigger signal and sends an acquisition command to the image acquisition module (2); The image acquisition module (2) acquires image sequences at a frame rate of not less than 120 frames per second according to the acquisition instruction, and transmits the original image data to the processing and control module (3) through the high-speed serial interface. The processing and control module (3) performs image enhancement processing on the original image data through its internal image signal processor (311), and compresses and encodes the processed image data into a video stream through its internal hardware encoder (312). The video stream is written into the DDR cache chip (4) for buffering; The buffered video stream is transmitted from the DDR cache chip (4) to the storage and transmission module (5) for local storage or network transmission.

10. A high-speed image capture method according to claim 9, characterized in that, The image enhancement processing of the original image data includes noise reduction, sharpening, and gamma correction; the compression encoding is hardware encoding using the H.264 or H.265 encoding standard.

Images