A smart high-speed vision camera and its imaging method for low-light imaging

By dividing the image into slices and processing them in parallel within a high-speed vision camera, and combining frequency domain illumination restoration and structural credibility guidance, the noise characteristics processing and real-time performance issues in low-light imaging of high-speed cameras are solved, achieving efficient low-light image enhancement and detail restoration.

CN123123745APending Publication Date: 2026-11-17CHANGCHUN INST OF OPTICS FINE MECHANICS & PHYSICS CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611601762.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-16
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies fail to effectively utilize imaging parameters in low-light imaging with high-speed vision cameras, resulting in improper noise characteristic processing, high frame processing latency, difficulty in meeting real-time requirements, insufficient frequency and spatial domain collaborative recovery, lack of reliable guidance in structural detail recovery, and excessive hardware deployment resource consumption.

Method used

By establishing a noise model, the image is divided into slices along the readout direction of the image sensor. The method of frequency domain illumination restoration and structural credibility guidance is adopted, combined with a hardware pipeline structure, to realize parallel processing and output of image slices, thereby reducing the buffer requirements of the whole frame.

Benefits of technology

It improves the imaging adaptability of low-light images, decouples brightness enhancement and detail restoration, reduces processing latency, and improves image naturalness and stability, making it suitable for high-speed, low-light, and low-latency visual imaging.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123745A_ABST
    Figure CN123123745A_ABST
Patent Text Reader

Abstract

This invention belongs to the field of image processing technology, and particularly relates to an intelligent high-speed vision camera and its imaging method for low-light imaging. The imaging method includes the following steps: S1, continuously acquiring low-light images to obtain continuous image data; outputting exposure time, analog gain, frame rate, and bit depth parameters; S2, establishing a noise model to obtain noise estimation; S3, dividing the continuous image data into multiple image slices; S4, based on a retinal imaging model, obtaining the initial illumination component and initial reflection component; and obtaining the amplitude component and phase component; and performing frequency-band enhancement on the amplitude component and bounded compensation on the phase component based on frequency domain confidence; S5, performing detail reconstruction on the frequency-restored reflection component; S6, performing weighted fusion on the overlapping areas of adjacent image blocks to output an enhanced image. This solution can achieve real-time enhancement and low-latency output of high-speed low-light images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to an intelligent high-speed vision camera for low-light imaging and its imaging method. Background Technology

[0002] With the continuous development of high-speed image sensors, intelligent image processing chips, and machine vision technology, high-speed vision cameras have been widely used in scenarios such as high-speed motion observation, industrial defect detection, 3D measurement, and target recognition. To improve the frame rate and reduce motion blur, high-speed cameras typically need to shorten the exposure time per frame. This results in a reduction in the number of effective photons received by the image sensor within the exposure cycle. Under low-light, weak-reflection, or high-speed motion conditions, the acquired images are prone to problems such as insufficient overall brightness, invisible targets in dark areas, reduced signal-to-noise ratio, prominent noise, and missing edge textures.

[0003] Traditional low-light image enhancement methods mainly include histogram equalization, gamma transform, and retinal enhancement. These methods primarily rely on fixed mathematical mappings to increase brightness, making it difficult to distinguish between real dark area structures and sensor noise, and easily leading to noise amplification, local overexposure, and detail distortion. For example, histogram equalization performs pixel equalization based on the gray values ​​of pixels statistically analyzed from the image histogram, without considering the relationships between neighboring pixels, often resulting in unnatural enhancement results. Retinal enhancement decomposes the image into reflectance and illumination components, but because the decomposition process is an ill-posed problem, it is difficult to completely separate and effectively enhance these components, and the enhancement effect significantly decreases under complex lighting conditions and noise.

[0004] In recent years, deep learning-based low-light image enhancement methods have become mainstream research. These methods automatically learn the transformation rules from low-light images to normally exposed images by constructing end-to-end nonlinear mapping relationships, achieving brightness enhancement and detail restoration to a certain extent simultaneously. Some methods further introduce attention mechanisms, multi-scale feature extraction, and physical constraints to improve the model's generalization ability and structure awareness. However, existing deep learning methods still have the following shortcomings: 1. Insufficient utilization of high-speed camera imaging parameters: Most existing methods process the image content directly without fully combining the imaging parameters of the high-speed camera, such as exposure time, analog gain, frame rate, and image bit depth. This makes it difficult for the enhancement process to adapt to the noise characteristics under short exposure and high gain conditions, and it is easy to cause noise amplification in dark areas and excessive enhancement in bright areas.

[0005] 2. The coupling of illumination restoration, noise suppression and structure reconstruction: Existing methods mostly use whole-frame depth networks for processing, which couples illumination restoration, noise suppression and detail reconstruction, easily resulting in overexposure in bright areas, amplification of noise in dark areas and false textures. At the same time, whole-frame caching and complex network calculations increase storage overhead and processing latency, making it difficult to meet the online imaging requirements of high-speed vision cameras.

[0006] 3. Insufficient coordination between frequency and spatial domains: Some existing methods only model illumination distribution in the spatial domain, ignoring the independent variation of frequency domain structure (low-frequency brightness and high-frequency texture), and lack a priori utilization mechanism for fixed viewpoints and long-term sequences, resulting in unstable enhancement effects in engineering applications; although some methods introduce frequency domain processing, they do not combine noise energy to build frequency domain confidence, which easily amplifies high-frequency components with low signal-to-noise ratio.

[0007] 4. Lack of credibility guidance in structural detail restoration: Existing methods often lack sufficient high-level semantic constraints in the image reconstruction process, resulting in problems such as texture distortion, edge blurring and structural artifacts in the enhancement results in complex structural regions, making it difficult to accurately restore the real scene information at the detail level; especially in areas with shadows, reflections or overlapping textures, the enhanced image may still have residual shadows and unclear outlines.

[0008] 5. Insufficient hardware deployment and real-time performance: Existing low-light enhancement methods mostly use whole-frame deep network processing, which requires large storage bandwidth and computing resources, and has high processing latency, making it difficult to meet the requirements of high-speed vision cameras for real-time performance, low latency and limited hardware resources; although directly using image block processing can reduce hardware resource consumption, it is easy to cause brightness discontinuity, texture breakage and splicing marks between adjacent blocks.

[0009] Specifically, the technical solutions in the prior art that are closest to this invention mainly include the following three categories: The first category is deep learning enhancement methods based on retinal theory. These methods unfold the iterative solution process of the retinal model into a neural network, or convert the optimization process of the reflection and illumination components into learnable iterative modules. They then use illumination estimation and illumination-guided recovery modules to address noise, artifacts, and color distortion exposed after brightening. While these methods enhance the interpretability and recovery capabilities of the retinal model, they process entire frames of images and are not specifically designed for the line-by-line readout, slice buffering, and hardware pipelines of high-speed cameras.

[0010] The second category is low-light restoration methods that coordinate frequency domain and spatial structure analysis. These methods employ a two-stage restoration framework. The first stage utilizes a residual Fourier-guided module to process amplitude and phase information, restoring global illumination. The second stage employs a tile state space module and a gradient state space module to enhance pixel details and high-gradient edge structures, respectively. This type of method demonstrates that restoring overall brightness in the frequency domain first, and then reconstructing local structures in the spatial domain, can balance visibility in dark areas with detail preservation. However, it still treats the entire image as a processing unit, failing to address noise modeling guided by high-speed camera imaging parameters, and also neglecting the issues of parallel processing in tiled blocks and low-latency output.

[0011] The third category is hardware-oriented low-light image enhancement solutions. These solutions deploy retinal enhancement algorithms on field-programmable gate arrays (FPGAs) and coarse-grained reconfigurable architectures, achieving low-latency processing through parallel computation and pipelined design, demonstrating that low-light enhancement algorithms can run in real-time on dedicated hardware. However, these solutions primarily implement traditional retinal computation, lacking the adaptive modeling of complex lighting, noise, and structural information by deep networks. They also fail to address issues such as full-frame caching of high-resolution, high-speed images, discontinuous tile boundaries, and parallel scheduling of acquisition, enhancement, and output.

[0012] In summary, although existing technologies have explored retinal depth modeling, frequency-domain-structure co-recovery, and hardware acceleration, there is still a lack of an intelligent high-speed vision camera and its imaging method for low-light imaging that can unify the design of high-speed camera imaging parameters, lightweight enhancement networks, parallel processing, boundary fusion, and hardware pipeline. Summary of the Invention

[0013] In view of this, the present invention aims to provide an intelligent high-speed vision camera and its imaging method for low-light imaging, in order to solve the problems of insufficient utilization of high-speed camera imaging parameters, mutual coupling of illumination restoration, noise suppression and structure reconstruction, high frame processing delay, and difficulty in meeting the online imaging requirements of high-speed vision cameras in the existing technology.

[0014] To achieve the above objectives, the technical solution created by this invention is implemented as follows: An intelligent high-speed vision camera imaging method for low-light imaging includes the following steps: S1. Continuously acquire low-light images using a high-speed image sensor to obtain continuous image data composed of multiple frames of original images; simultaneously output exposure time, analog gain, frame rate, and bit depth parameters. S2. Establish a noise model based on exposure time, analog gain, frame rate, and bit depth parameters, and perform black level compensation and bit depth normalization on low-light images to obtain noise estimates. S3. Along the row readout direction of the image sensor, divide the continuous image data into multiple image slices, and each image slice is divided into image blocks with overlapping areas; S4. Based on the retinal imaging model, frequency domain illumination recovery is performed on the image patch to obtain the initial illumination component and the initial reflection component; shallow features of the initial reflection component are extracted, and amplitude and phase decomposition is performed on the shallow features to obtain amplitude components and phase components; frequency domain confidence is constructed based on amplitude energy and noise energy, and frequency band enhancement is performed on the amplitude component and bounded compensation is performed on the phase component based on the frequency domain confidence; the enhanced amplitude component and the compensated phase component are recombined and inverse transform is performed, and the frequency domain recovered reflection component is obtained through feature reconstruction. S5. Based on structural credibility, detailed reconstruction of the reflection components after frequency domain restoration is performed. Structural credibility is jointly determined by gradient information, local variance, illumination components and noise estimation. S6. Weighted fusion is performed on the overlapping areas of adjacent image blocks, and the enhanced image is output in units of image slices through a hardware pipeline.

[0015] Furthermore, the noise model established in S2 is as follows: The exposure time, analog gain, frame rate, and bit depth are combined to form an imaging parameter vector, which is expressed by the formula: ; in, Represents the imaging parameter vector; Indicates the exposure time; Indicates the analog gain; Indicates frame rate; Indicates the image bit depth; A noise model is established, expressed by the following formula: ; in, Represent the noise model; Represents the signal-dependent noise figure; Indicates the brightness of the input image; This indicates the readout noise figure.

[0016] Furthermore, in S4, frequency domain illumination recovery is performed on the image patch, specifically including: S41. Represent the image according to the retinal imaging model. The initial illumination components are estimated using a lightweight convolutional network to obtain the initial reflection components; where, Indicates the input image; Indicates the reflected component; Indicates the lighting component; This represents the noise term in the image decomposition model; This indicates element-wise multiplication; S42. Perform a two-dimensional fast Fourier transform on the shallow features to obtain the amplitude component and the phase component. S43. Construct frequency domain confidence based on amplitude energy and noise energy; S44. Divide the frequency domain into low-frequency, mid-frequency, and high-frequency regions, and predict the gain of each frequency band by combining imaging parameters to enhance the amplitude component. S45. Bounded phase compensation is used to correct the phase components; S46. Recombine the enhanced amplitude component and the compensated phase component and perform an inverse transform to obtain the frequency domain recovery feature; perform feature reconstruction on the frequency domain recovery feature to obtain the frequency domain recovered reflection component.

[0017] Furthermore, in S43, the frequency domain confidence level is specifically as follows: ; in, Indicates frequency domain confidence; Indicates amplitude components; The frequency domain representation of the noise term; This represents a constant that prevents division by zero.

[0018] Furthermore, in S44, the amplitude component is enhanced as follows: ; in, This represents the enhanced amplitude component; Indicates amplitude components; Indicates frequency domain confidence; , , These represent low-frequency masks, mid-frequency masks, and high-frequency masks, respectively. , These represent low-frequency gain and mid-frequency gain, respectively. Indicates the high-frequency noise suppression coefficient; This indicates element-wise multiplication.

[0019] Furthermore, in S45, the bounded phase compensation specifically refers to: ; in, This represents the compensated phase component; Indicates the phase component; Indicates frequency domain confidence; Indicates the upper limit of phase compensation; Represents the phase compensation function; Indicates amplitude components; This indicates element-wise multiplication.

[0020] Furthermore, in S5, detailed reconstruction of the frequency-domain recovered reflection components is performed based on structural reliability, specifically including: S51. Calculate the horizontal gradient, vertical gradient, Laplace response, and local variance of the frequency domain recovered reflection components obtained in S4. S52. Input the gradient, local variance, illumination component, and noise estimation into the confidence prediction network to generate a structural confidence map. S53. Predict structural residuals and noise residuals using a lightweight structural recovery network; S54. The structural residuals and noise residuals are adjusted using the structural reliability diagram to obtain the enhanced reflection component. S55. The enhanced reflection component is fused with the illumination component obtained in S4 to obtain the final enhanced image.

[0021] In S54, the enhanced reflection component is specifically as follows: ; in, This represents the enhanced reflection component; Indicates the initial reflection component; Represents the structural credibility graph; Indicates structural residuals; Indicates noise residual; Indicates the maximum value of the reflected component; Indicates a truncation operation; This indicates element-wise multiplication.

[0022] Furthermore, in S6, the weighted fusion of overlapping regions of adjacent image blocks is specifically performed as follows: ; in, This represents the merged pixel values; Indicates the covered pixels A collection of image patches; Represents image blocks In pixels The fusion weight at the location; Represents image blocks In pixels Pixel value at; A constant that prevents division by zero; Indicates pixel position; Indicates the image patch index.

[0023] A smart high-speed vision camera for low-light imaging, used to execute the aforementioned smart high-speed vision camera imaging method for low-light imaging, includes: The high-speed image acquisition and imaging modeling module is used to execute S1 and S2; The fragmented scheduling module is used to execute S3; Frequency domain illumination recovery module, used to execute S4; The structural detail recovery module is used to execute S5; The hardware acceleration module is used to execute the hardware pipeline in S6, and realizes the parallel execution of image acquisition, enhancement calculation and result output through a segmented and block-based parallel pipeline structure. The image output module is used to perform the output steps in S6, outputting enhanced images in units of image slices through a hardware pipeline.

[0024] Compared with the prior art, the present invention can achieve the following beneficial effects: 1. Improve the imaging adaptability of high-speed, short-exposure, low-light images: This invention introduces exposure time, analog gain, image bit depth, and noise characteristics into the low-light enhancement process, which can adaptively adjust the enhancement intensity according to the actual imaging state of the high-speed camera. Compared with methods that only process the image content, this invention is more suitable for high-speed, low-light imaging under short-exposure, high-gain conditions, and can reduce noise amplification in dark areas and excessive enhancement in bright areas.

[0025] 2. Decoupling of overall brightness enhancement and local detail restoration: This invention adopts a processing method of first restoring the frequency domain illumination and then reconstructing the spatial structure, thereby decoupling the overall brightness enhancement and local detail restoration. The frequency domain module uses noise confidence and frequency band gain to restore global illumination and suppress high-frequency noise, while the structure restoration module uses structural confidence to reconstruct real edges and textures, thus taking into account the visibility of dark areas, noise suppression and detail clarity.

[0026] 3. Improve the naturalness and stability of enhanced images: This invention uses gradient, local variance, illumination and noise information to jointly determine structural reliability, avoiding enhancement of details based solely on high gradient responses. Compared with direct sharpening or uniform enhancement methods, it can reduce random noise, bad pixel responses and the situation where false edges are mistakenly restored as textures, thereby improving the naturalness and stability of enhanced images.

[0027] 4. Reduced frame buffering requirements and end-to-end processing latency: This invention adopts segmented and block-based streaming processing, overlapping region fusion, and hardware pipeline structure. Enhanced computation can be started without waiting for the entire frame image to be acquired. Compared with the whole frame deep network processing method, it can reduce off-chip storage access and frame buffering capacity, improve computing resource utilization, reduce processing latency, and is easy to implement in field programmable gate arrays, on-chip systems, or embedded intelligent processors.

[0028] 5. Providing an effective technical solution for high-speed, low-light, and low-latency visual imaging: This invention realizes the collaborative design of low-light enhancement network, image signal processing, and hardware parallel architecture, which can balance low-light enhancement quality, hardware resource utilization, and real-time processing performance, and can provide a stable, clear, and easy-to-deploy technical solution for high-speed, low-light, and low-latency visual imaging. Attached Figure Description

[0029] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is an overall flowchart of the intelligent high-speed vision camera imaging method in an embodiment of the present invention. Detailed Implementation

[0030] To make the purpose, technical solution, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1 The present invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and do not constitute a limitation thereof.

[0031] This invention provides an intelligent high-speed vision camera imaging method for low-light imaging, comprising the following steps: S1. Continuously acquire low-light images through a high-speed image sensor to obtain continuous image data composed of multiple frames of original images; at the same time, output exposure time, analog gain, frame rate, and bit depth parameters.

[0032] The original image is represented by the formula: ; in, This represents the raw image output by the high-speed camera; Indicates the quantification process; Indicates the analog gain; Indicates the reflected component; Indicates the lighting component; Indicates shot noise; Indicates readout noise; Indicates black level; This indicates element-wise multiplication.

[0033] S2. Based on the exposure time, analog gain, frame rate, and bit depth parameters, a noise model is established, and black level compensation and bit depth normalization are performed on the low-light image to obtain the noise estimate.

[0034] The noise model established is as follows: The exposure time, analog gain, frame rate, and bit depth are combined to form an imaging parameter vector, which is expressed by the formula: ; in, Represents the imaging parameter vector; Indicates the exposure time; Indicates the analog gain; Indicates frame rate; Indicates the image bit depth.

[0035] A noise model is established, expressed by the following formula: ; in, Represent the noise model; Represents the signal-dependent noise figure; Indicates the brightness of the input image; This indicates the readout noise figure.

[0036] S3. Along the row readout direction of the image sensor, divide the continuous image data into multiple image slices, and each image slice is divided into image blocks with overlapping areas.

[0037] S4. Based on the retinal imaging model, frequency domain illumination recovery is performed on the image patch to obtain the initial illumination component and the initial reflection component; shallow features of the initial reflection component are extracted, and amplitude and phase decomposition is performed on the shallow features to obtain amplitude components and phase components; frequency domain confidence is constructed based on amplitude energy and noise energy, and frequency band enhancement is performed on the amplitude component and bounded compensation is performed on the phase component based on the frequency domain confidence; the enhanced amplitude component and the compensated phase component are recombined and inverse transform is performed, and the frequency domain recovered reflection component is obtained through feature reconstruction.

[0038] Frequency domain illumination restoration of image patches specifically includes: S41. Represent the image according to the retinal imaging model. ;in, Indicates the input image; Indicates the reflected component; Indicates the lighting component; This represents the noise term in the image decomposition model; This indicates element-wise multiplication.

[0039] The initial illumination components are estimated using a lightweight convolutional network, and the initial reflection components are obtained as follows: ; in, Indicates the initial reflection component; Indicates a truncation operation; This represents the low-light image after black level compensation and bit depth normalization in S2; This represents the initial illumination components estimated using a lightweight convolutional network. A constant that prevents division by zero; This indicates the maximum value of the reflection component.

[0040] S42. Perform a two-dimensional fast Fourier transform on the shallow features, expressed by the formula: ; in, This represents the result of the frequency domain transformation. Indicates shallow features; Indicates shallow features Perform a two-dimensional fast Fourier transform; It represents the amplitude component, mainly describing brightness and contrast; The complex number representation in the frequency domain; j represents the imaginary unit; This represents the phase component.

[0041] S43. Construct frequency domain confidence based on amplitude energy and noise energy.

[0042] The frequency domain confidence level is specifically as follows: ; in, Indicates frequency domain confidence; Indicates amplitude components; The frequency domain representation of the noise term; This represents a constant that prevents division by zero.

[0043] S44. Divide the frequency domain into low-frequency, mid-frequency, and high-frequency regions, and predict the gain of each frequency band based on imaging parameters to enhance the amplitude component.

[0044] The specific steps to enhance the amplitude component are as follows: ; in, This represents the enhanced amplitude component; Indicates amplitude components; Indicates frequency domain confidence; , , These represent low-frequency masks, mid-frequency masks, and high-frequency masks, respectively. , These represent low-frequency gain and mid-frequency gain, respectively. Indicates the high-frequency noise suppression coefficient; This indicates element-wise multiplication.

[0045] S45. Bounded phase compensation is used to correct the phase components.

[0046] Bounded phase compensation specifically refers to: ; in, This represents the compensated phase component; Indicates the phase component; Indicates frequency domain confidence; Indicates the upper limit of phase compensation; Represents the phase compensation function; Indicates amplitude components; This indicates element-wise multiplication.

[0047] S46. Recombine the enhanced amplitude component and the compensated phase component and perform an inverse transform to obtain the frequency domain recovery feature; perform feature reconstruction on the frequency domain recovery feature to obtain the frequency domain recovered reflection component.

[0048] The inverse transform can be expressed by the following formula: ; in, This represents the initial enhanced image in the frequency domain stage; This indicates taking the real part of a complex number; This represents the two-dimensional inverse fast Fourier transform. This represents the enhanced amplitude component; The complex number representation of the phase component after compensation; This represents the compensated phase component; j represents the imaginary unit. This indicates shallow features.

[0049] S5. Based on structural credibility, detailed reconstruction of the reflection components after frequency domain restoration is performed. Structural credibility is jointly determined by gradient information, local variance, illumination components and noise estimation.

[0050] Based on structural reliability, detailed reconstruction of the reflection components after frequency domain restoration is performed, specifically including: S51. Calculate the horizontal gradient, vertical gradient, Laplace response, and local variance of the frequency domain recovered reflection component obtained in S4.

[0051] Specifically, the formula is expressed as: ; ; in, Indicates the gradient magnitude; Represents the horizontal gradient; Represents the vertical gradient; A constant that prevents division by zero; Indicates local variance; Represented in pixels The local area centered on; express Pixel position index within; This indicates the reflection component after frequency domain reconstruction at the pixel level. Pixel value at; express The average pixel value within the range.

[0052] S52. Input the gradient, local variance, illumination component, and noise estimation into the confidence prediction network to generate a structural confidence map.

[0053] Specifically, the formula is expressed as: ; in, Represents the structural credibility graph; This represents the Sigmoid activation function; The mapping function representing the credibility prediction network; Indicates the gradient magnitude; Represents the gradient of the illumination component; Indicates local variance; Indicates the initial illumination component; This indicates noise estimation.

[0054] Among them, real continuous edges and textured regions Areas with higher elevations, stronger noise levels, or insufficient structural support Lower.

[0055] S53. Predict structural residuals and noise residuals using a lightweight structural recovery network.

[0056] S54. The structural residuals and noise residuals are adjusted using the structural reliability diagram to obtain the enhanced reflection component.

[0057] The lightweight structure recovery network consists of channel-wise convolution, pointwise convolution, and residual connections. It avoids multi-level downsampling to reduce detail loss and hardware caching requirements. The enhanced reflection components are specifically as follows: ; in, This represents the enhanced reflection component; Indicates the initial reflection component; Represents the structural credibility graph; Indicates structural residuals; Indicates noise residual; Indicates the maximum value of the reflected component; Indicates a truncation operation; This indicates element-wise multiplication.

[0058] S55. The enhanced reflection component is fused with the illumination component obtained in S4 to obtain the final enhanced image, expressed by the formula: ; in, This represents the final enhanced image; Indicates a truncation operation; This represents the enhanced reflection component; This indicates element-wise multiplication; This represents the illumination component after frequency domain recovery.

[0059] S6. Weighted fusion of overlapping areas of adjacent image blocks is performed, and enhanced images are output in units of image slices through a hardware pipeline. By fusing overlapping areas, brightness abrupt changes and texture breaks between image blocks can be reduced.

[0060] The weighted fusion of overlapping regions of adjacent image patches is specifically performed as follows: ; in, This represents the merged pixel values; Indicates the covered pixels A collection of image patches; Represents image blocks In pixels The fusion weight at the location; Represents image blocks In pixels Pixel value at; A constant that prevents division by zero; Indicates pixel position; Indicates the image patch index.

[0061] The hardware system includes a high-speed sensor interface, line buffer, sliced ​​double-buffered memory, image block scheduler, frequency domain operation unit, convolution operation unit, boundary fusion unit, and image output interface. It adopts a double-buffered and pipelined structure, which enables the current slice acquisition, the previous slice enhancement, and the previous slice output to be executed in parallel, without waiting for the entire frame image to be acquired before starting processing.

[0062] Network weights are stored using fixed-point quantization. Channel-wise convolution, point-wise convolution, and frequency domain operations are implemented through parallel multiply-accumulate arrays, pipelined butterfly operations, and on-chip caches, respectively, thereby reducing data transfer volume, storage resources, and end-to-end processing latency.

[0063] A smart high-speed vision camera for low-light imaging, used to execute the aforementioned smart high-speed vision camera imaging method for low-light imaging, includes: The high-speed image acquisition and imaging modeling module is used to execute S1 and S2. It continuously acquires low-light images through a high-speed image sensor to obtain continuous image data composed of multiple frames of original images, and outputs exposure time, analog gain, frame rate, and bit depth parameters. Based on the exposure time, analog gain, frame rate, and bit depth parameters, a noise model is established, and black level compensation and bit depth normalization are performed on the low-light images to obtain noise estimation.

[0064] The slice and block scheduling module is used to execute S3, which divides continuous image data into multiple image slices along the row readout direction of the image sensor, and each image slice is divided into image blocks with overlapping areas.

[0065] The frequency domain illumination recovery module is used to execute S4, which performs frequency domain illumination recovery on the image patch based on the retinal imaging model to obtain the illumination component and the reflection component; extracts the shallow features of the image patch and performs amplitude and phase decomposition on the shallow features to obtain the amplitude component and the phase component; constructs a frequency domain confidence score based on the amplitude energy and noise energy, and performs frequency band enhancement on the amplitude component and bounded compensation on the phase component based on the frequency domain confidence score.

[0066] The structural detail recovery module is used to execute S5, which performs detail reconstruction on the frequency domain recovered reflection components based on structural confidence. The structural confidence is jointly determined by gradient information, local variance, illumination components and noise estimation.

[0067] The hardware acceleration module is used to execute the hardware pipeline in S6, and realizes the parallel execution of image acquisition, enhancement calculation and result output through a segmented and block-based parallel pipeline structure.

[0068] The image output module is used to perform the output steps in S6, perform weighted fusion on the overlapping areas of adjacent image blocks, and output the enhanced image in units of image slices through the hardware pipeline.

[0069] It should be noted that the high-speed camera imaging parameter modeling, frequency domain illumination restoration, structural reliability guidance, segmentation and block processing, and hardware pipeline structure in this invention can not only be used for low-light enhancement of high-speed vision cameras, but also applied to scenarios such as industrial inspection, night monitoring, high-speed motion analysis, dark field microscopy imaging, three-dimensional vision measurement, and low-light target recognition.

[0070] In one alternative implementation, the retinal decomposition in the frequency domain illumination recovery module can be replaced by explicit illumination-reflection decomposition, algorithmic unfolding network, or implicit illumination estimation network.

[0071] In one alternative implementation, frequency domain processing can be achieved using Fourier transform, discrete cosine transform, wavelet transform, or other multi-band decomposition methods.

[0072] In one alternative implementation, structural credibility can be constructed using gradient, edge, texture, signal-to-noise ratio, or neural network prediction methods.

[0073] In one alternative implementation, the lightweight structure recovery network can be replaced with ordinary convolution, depthwise separable convolution, state space unit, or lightweight attention unit.

[0074] In one alternative implementation, the segmentation method can be along the row direction, column direction, or time sequence direction; overlapping regions can be fused using cosine weights, Gaussian weights, triangular weights, or learnable weights.

[0075] In one alternative implementation, the hardware platform may be implemented using a field-programmable gate array, a system-on-a-chip, an application-specific integrated circuit, a graphics processor, or a neural network processor.

[0076] In one alternative implementation, the hardware acceleration module includes a high-speed sensor interface, a line cache, a sliced ​​double buffer memory, an image block scheduler, a frequency domain operation unit, a convolution calculation unit, a boundary fusion unit, and an image output interface.

[0077] The high-speed sensor interface receives continuous image data from the high-speed image sensor and transmits the continuous image data to the segmented double-buffered memory via row buffering; the image block scheduler reads image slices and image blocks from the segmented double-buffered memory and schedules the image blocks to the frequency domain operation unit and the convolution calculation unit, respectively; the frequency domain operation unit performs frequency domain illumination restoration; the convolution calculation unit performs convolution operations in structural detail restoration; the boundary fusion unit performs weighted fusion of overlapping areas of adjacent image blocks; and the image output interface outputs the enhanced image.

[0078] The system adopts a dual-buffered and pipelined structure, which enables the current segment acquisition, the previous segment enhancement, and the previous segment output to be executed in parallel, without waiting for the entire frame image to be acquired before starting processing.

[0079] It should be noted that the above alternative solutions are merely illustrative examples. Any implementation method that can achieve coordinated restoration of illumination and structure of low-light images, continuous block boundaries, and low-latency parallel operation of acquisition, processing, and output can accomplish the purpose of this invention and falls within the protection scope of this invention.

[0080] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0081] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An intelligent high-speed vision camera imaging method for low-light imaging, characterized in that, Includes the following steps: S1. Continuously acquire low-light images using a high-speed image sensor to obtain continuous image data composed of multiple frames of original images; simultaneously output exposure time, analog gain, frame rate, and bit depth parameters. S2. Establish a noise model based on exposure time, analog gain, frame rate, and bit depth parameters, and perform black level compensation and bit depth normalization on low-light images to obtain noise estimates. S3. Along the row readout direction of the image sensor, divide the continuous image data into multiple image slices, and each image slice is divided into image blocks with overlapping areas; S4. Based on the retinal imaging model, frequency domain illumination recovery is performed on the image patch to obtain the initial illumination component and the initial reflection component; shallow features of the initial reflection component are extracted, and amplitude and phase decomposition is performed on the shallow features to obtain amplitude components and phase components; frequency domain confidence is constructed based on amplitude energy and noise energy, and frequency band enhancement is performed on the amplitude component and bounded compensation is performed on the phase component based on the frequency domain confidence; the enhanced amplitude component and the compensated phase component are recombined and inverse transform is performed, and the frequency domain recovered reflection component is obtained through feature reconstruction. S5. Based on structural credibility, detailed reconstruction of the reflection components after frequency domain restoration is performed. Structural credibility is jointly determined by gradient information, local variance, illumination components and noise estimation. S6. Weighted fusion is performed on the overlapping areas of adjacent image blocks, and the enhanced image is output in units of image slices through a hardware pipeline.

2. The intelligent high-speed vision camera imaging method for low-light imaging according to claim 1, characterized in that, The noise model established in S2 is as follows: The exposure time, analog gain, frame rate, and bit depth are combined to form an imaging parameter vector, which is expressed by the formula: ; in, Represents the imaging parameter vector; Indicates the exposure time; Indicates the analog gain; Indicates frame rate; Indicates the image bit depth; A noise model is established, expressed by the following formula: ; in, Represent the noise model; Represents the signal-dependent noise figure; Indicates the brightness of the input image; This indicates the readout noise figure.

3. The intelligent high-speed vision camera imaging method for low-light imaging according to claim 1, characterized in that, In S4, frequency domain illumination restoration of the image patch is performed, specifically including: S41. Represent the image according to the retinal imaging model. The initial illumination components are estimated using a lightweight convolutional network to obtain the initial reflection components; where, Indicates the input image; Indicates the reflected component; Indicates the lighting component; This represents the noise term in the image decomposition model; This indicates element-wise multiplication; S42. Perform a two-dimensional fast Fourier transform on the shallow features to obtain the amplitude component and the phase component. S43. Construct frequency domain confidence based on amplitude energy and noise energy; S44. Divide the frequency domain into low-frequency, mid-frequency, and high-frequency regions, and predict the gain of each frequency band by combining imaging parameters to enhance the amplitude component. S45. Bounded phase compensation is used to correct the phase components; S46. Recombine the enhanced amplitude component and the compensated phase component and perform an inverse transform to obtain the frequency domain recovery feature; perform feature reconstruction on the frequency domain recovery feature to obtain the frequency domain recovered reflection component.

4. The intelligent high-speed vision camera imaging method for low-light imaging according to claim 3, characterized in that, In S43, the frequency domain confidence level is specifically as follows: ; in, Indicates frequency domain confidence; Indicates amplitude components; Frequency domain representation of the noise term; This represents a constant that prevents division by zero.

5. The intelligent high-speed vision camera imaging method for low-light imaging according to claim 4, characterized in that, In S44, the amplitude component is enhanced as follows: ; in, This represents the enhanced amplitude component; Indicates amplitude components; Indicates frequency domain confidence; , , These represent low-frequency masks, mid-frequency masks, and high-frequency masks, respectively. , These represent low-frequency gain and mid-frequency gain, respectively. Indicates the high-frequency noise suppression coefficient; This indicates element-wise multiplication.

6. The intelligent high-speed vision camera imaging method for low-light imaging according to claim 5, characterized in that, In S45, the bounded phase compensation is specifically as follows: ; in, This represents the compensated phase component; Indicates the phase component; Indicates frequency domain confidence; Indicates the upper limit of phase compensation; Represents the phase compensation function; Indicates amplitude components; This indicates element-wise multiplication.

7. The intelligent high-speed vision camera imaging method for low-light imaging according to claim 1, characterized in that, In S5, detailed reconstruction of the frequency-domain recovered reflection components is performed based on structural reliability, specifically including: S51. Calculate the horizontal gradient, vertical gradient, Laplace response, and local variance of the frequency domain recovered reflection components obtained in S4. S52. Input the gradient, local variance, illumination component, and noise estimation into the confidence prediction network to generate a structural confidence map. S53. Predict structural residuals and noise residuals using a lightweight structural recovery network; S54. The structural residuals and noise residuals are adjusted using the structural reliability diagram to obtain the enhanced reflection component. S55. The enhanced reflection component is fused with the illumination component obtained in S4 to obtain the final enhanced image.

8. The intelligent high-speed vision camera imaging method for low-light imaging according to claim 7, characterized in that, In S54, the enhanced reflection component is specifically as follows: ; in, This represents the enhanced reflection component; Indicates the initial reflection component; Represents the structural credibility graph; Indicates structural residuals; Indicates noise residual; Indicates the maximum value of the reflected component; Indicates a truncation operation; This indicates element-wise multiplication.

9. The intelligent high-speed vision camera imaging method for low-light imaging according to claim 1, characterized in that, In S6, the weighted fusion of overlapping regions of adjacent image blocks is specifically performed as follows: ; in, This represents the merged pixel values; Indicates the covered pixels A collection of image patches; Represents image blocks In pixels The fusion weight at the location; Represents image blocks In pixels Pixel value at; A constant that prevents division by zero; Indicates pixel position; Indicates the image patch index.

10. A smart high-speed vision camera for low-light imaging, used to execute the imaging method of the smart high-speed vision camera for low-light imaging as described in any one of claims 1-9, characterized in that, include: The high-speed image acquisition and imaging modeling module is used to execute S1 and S2; The fragmented scheduling module is used to execute S3; Frequency domain illumination recovery module, used to execute S4; The structural detail recovery module is used to execute S5; The hardware acceleration module is used to execute the hardware pipeline in S6, and realizes the parallel execution of image acquisition, enhancement calculation and result output through a segmented and block-based parallel pipeline structure. The image output module is used to perform the output steps in S6, outputting enhanced images in units of image slices through a hardware pipeline.