Motion compensation single-pixel real-time imaging method and system for dynamic scene

Abstract

The application provides a motion compensation single-pixel real-time imaging method and system for a dynamic scene, and belongs to the technical field of scene imaging. The method comprises the following steps: projecting a set of geometric moment modulation patterns subjected to differential normalization to a dynamic scene containing a moving target, so as to obtain a time-series light intensity signal; the geometric moment modulation patterns and Hadamard modulation patterns used for image reconstruction are staggered and arranged according to a preset ratio to form a composite modulation sequence; the center-of-mass coordinate and the pre-rotation angle of the moving target in a current reconstruction frame segment are calculated from the time-series light intensity signal; the motion offset parameter formed by the center-of-mass coordinate and the rotation angle is used to determine a reconstruction modulation pattern after motion compensation; and the time-series light intensity signal and the reconstruction modulation pattern after motion compensation are subjected to correlation calculation, so as to obtain a dynamic scene image sequence in which motion blur is eliminated. The technical scheme provided by the application can avoid the correlation reconstruction blur problem caused by the mismatch between the modulation pattern and the spatial position of the target due to the non-cooperative moving target in the dynamic scene.

Description

Technical Field

[0001] This application relates to the field of scene imaging technology, and more specifically, to a motion-compensated single-pixel real-time imaging method and system for dynamic scenes. Background Technology

[0002] In the field of computer vision and graphics, scene imaging aims to convert three-dimensional scene information into two-dimensional images through physical simulation and mathematical modeling. Traditional rendering techniques such as rasterization and ray tracing can generate highly realistic images, but they are computationally expensive, especially when dealing with complex optical effects such as global illumination and soft shadows. In recent years, with the rise of deep learning and neural rendering, high-quality scene imaging is being promoted for widespread application in fields such as virtual reality, digital twins, and film and television production.

[0003] In existing scene imaging, the imaging process begins with light emitting from a light source. After the light shines on the surface of an object in the 3D scene, it is reflected, refracted, or scattered. Some of the light carrying information about the object's surface propagates along a specific direction and eventually enters the aperture of the imaging device. Subsequently, the imaging system uses a lens group to converge and refocus the collected light, forming an inverted, reduced real image on the photosensitive element or focal plane. However, in motion-compensated single-pixel real-time imaging for dynamic scenes, when there are non-cooperative moving targets in the dynamic scene, the spatial position and orientation of the target change continuously during the successive projection of the modulation pattern. This causes a misalignment between the modulation pattern, which was originally aligned with the static scene, and the dynamically changing reflection area on the target. As a result, during the time accumulation of conventional single-pixel imaging sampling, the spatial position and orientation of the moving target in the dynamic scene drift within the frame, leading to a mismatch in the spatiotemporal mapping relationship between the Hadamard modulation pattern and the actual spatial distribution of the target. This, in turn, causes geometric distortion and motion blur in the reconstructed image. Therefore, how to avoid the correlation reconstruction blur caused by the mismatch between the modulation pattern and the target's spatial position due to non-cooperative moving targets in dynamic scenes has become a challenge for the industry. Summary of the Invention

[0004] This application provides a motion-compensated single-pixel real-time imaging method and system for dynamic scenes, which can avoid the problem of blurred correlation reconstruction caused by the mismatch between the modulation pattern and the spatial position of the target due to non-cooperative moving targets in dynamic scenes.

[0005] In a first aspect, this application provides a motion-compensated single-pixel real-time imaging method for dynamic scenes, comprising the following steps:

[0006] A set of geometric moment modulation patterns, after differential normalization processing, is projected onto a dynamic scene containing moving targets, and the temporal light intensity signal reflected back from the dynamic scene is collected. The geometric moment modulation patterns and the Hadamard modulation patterns used for image reconstruction are interleaved in a preset ratio to form a composite modulation sequence.

[0007] The zero-order moment, first-order moment, and second-order central moment corresponding to the current reconstructed frame are calculated from the time-series light intensity signal. The centroid coordinates of the moving target in the current reconstructed frame are determined based on the zero-order moment and first-order moment after background compensation. The rotation angle of the moving target in the current reconstructed frame is determined based on the centroid coordinates and the second-order central moment obtained by normalized difference processing.

[0008] The Hadamard modulation pattern of the current reconstructed frame segment in the composite modulation sequence is reverse-shifted using the motion offset parameter formed by the centroid coordinates and the rotation angle to generate a motion-compensated reconstructed modulation pattern.

[0009] The temporal light intensity signal and the motion-compensated reconstructed modulation pattern are correlated and calculated to reconstruct a dynamic scene image sequence with motion blur eliminated.

[0010] In some embodiments, projecting a set of geometric moment modulation patterns, after differential normalization processing, onto a dynamic scene containing moving targets, and acquiring the temporal light intensity signal reflected back from the dynamic scene specifically includes:

[0011] A set of geometric moment modulation patterns is determined by projection after differential normalization, the geometric moment modulation patterns including a first-order moment modulation pattern after normalized differential processing and a second-order central moment modulation pattern after normalized differential processing;

[0012] The geometric moment modulation pattern is loaded onto a spatial light modulator, and the geometric moment modulation pattern is projected onto a dynamic scene containing moving targets via the spatial light modulator;

[0013] The time-series light intensity signal reflected back from the dynamic scene is acquired by a single-pixel detector.

[0014] In some embodiments, determining a set of geometric moment modulation patterns that have undergone differential normalization specifically includes:

[0015] A two-dimensional gray-level basis function matching the resolution of the spatial light modulator is generated, and the two-dimensional gray-level basis function is integrated in the corresponding order space to obtain a first-order gray-level moment basis pattern and a second-order gray-level central moment basis pattern.

[0016] The absolute values ​​of the positive and negative parts in the grayscale first-order moment base pattern are normalized to their maximum values ​​to obtain a first-order moment positive differential modulation sub-pattern and a first-order moment negative differential modulation sub-pattern, thereby obtaining a normalized differential first-order moment modulation pattern.

[0017] The absolute values ​​of the positive and negative parts in the grayscale second-order central moment base pattern are normalized to their maximum values ​​to obtain a second-order central moment positive differential modulation sub-pattern and a second-order central moment negative differential modulation sub-pattern, thereby obtaining a normalized differential second-order central moment modulation pattern.

[0018] A geometric moment modulation pattern is obtained by combining a first-order moment modulation pattern processed by normalized differential processing and a second-order central moment modulation pattern processed by normalized differential processing.

[0019] In some embodiments, the geometric moment modulation pattern and the Hadamard modulation pattern used for image reconstruction are interleaved at a preset ratio to form a composite modulation sequence, specifically including:

[0020] The total number of Hadamard modulation patterns and the total number of geometric moment modulation patterns required to be included in the current reconstructed frame segment are determined according to the preset composite interleaving ratio.

[0021] A corresponding number of Hadamard base patterns are generated according to the total number of Hadamard modulation patterns, and the Hadamard base patterns are sorted in order of energy concentration from high to low to obtain a Hadamard sorting sequence.

[0022] The geometric moment modulation pattern is used as a geometric moment measurement unit, and the geometric moment measurement unit is inserted into the Hadamard sorting sequence at equal intervals. Then, the Hadamard sorting sequence after inserting the geometric moment measurement unit is determined as the composite modulation sequence.

[0023] In some embodiments, calculating the zeroth moment, first moment, and second central moment corresponding to the current reconstructed frame segment from the temporal light intensity signal specifically includes:

[0024] Extract four probe light intensity values ​​corresponding to the four differential modulation sub-patterns contained in the geometric moment measurement unit within the current reconstructed frame segment from the time-series light intensity signal. The four probe light intensity values ​​are composed of a first positive light intensity value, a first negative light intensity value, a second positive light intensity value, and a second negative light intensity value.

[0025] Perform a difference operation between the first positive light intensity value and the first negative light intensity value to obtain a first-order moment difference detection value, and determine the first-order moment difference detection value as the first-order moment value corresponding to the current reconstructed frame segment;

[0026] Perform a difference operation between the second positive light intensity value and the second negative light intensity value to obtain the second-order central moment difference detection value, and determine the second-order central moment difference detection value as the second-order central moment value corresponding to the current reconstructed frame segment;

[0027] Extract the light intensity detection value corresponding to the fully bright pattern in the Hadamard modulation pattern within the current reconstructed frame segment, and determine the light intensity detection value corresponding to the fully bright pattern as the zero-order moment value corresponding to the current reconstructed frame segment.

[0028] In some embodiments, determining the centroid coordinates of the moving target in the current reconstructed frame segment based on the zeroth and first moments after background compensation specifically includes:

[0029] Subtract the pre-calibrated static background zero-moment value from the zero-moment value corresponding to the current reconstructed frame segment to obtain the zero-moment value after background compensation.

[0030] The first moment value corresponding to the current reconstructed frame segment is compared with the zero moment value after background compensation in a dimension-wise ratio operation to obtain the centroid abscissa and centroid ordinate of the moving target in the current reconstructed frame segment.

[0031] The centroid coordinates of the moving target in the current reconstructed frame are formed by the centroid abscissa and the centroid ordinate.

[0032] In some embodiments, the correlation calculation between the temporal light intensity signal and the motion-compensated reconstructed modulation pattern to reconstruct a dynamic scene image sequence with motion blur eliminated specifically includes:

[0033] Extract the sequence of Hadamard probe light intensity values ​​corresponding to the Hadamard modulation patterns of each frame in the current reconstructed frame segment from the time-series light intensity signal;

[0034] Perform an inner product operation between each probe light intensity value in the sequence of Hadamard probe light intensity values ​​and the modulation pattern of the corresponding frame in the motion-compensated reconstructed modulation pattern to obtain the one-dimensional Hadamard transform coefficient vector corresponding to the current reconstructed frame segment.

[0035] Perform an inverse Hadamard transform on the one-dimensional Hadamard transform coefficient vector to reconstruct a single frame image corresponding to the reconstructed frame segment in the dynamic scene image sequence with motion blur eliminated.

[0036] Secondly, this application provides a motion-compensated single-pixel real-time imaging system for dynamic scenes, used to execute a motion-compensated single-pixel real-time imaging method for dynamic scenes, the system comprising:

[0037] The acquisition module is used to project a set of geometric moment modulation patterns that have undergone differential normalization processing onto a dynamic scene containing moving targets, and to acquire the temporal light intensity signal reflected back from the dynamic scene. The geometric moment modulation patterns and the Hadamard modulation patterns used for image reconstruction are interleaved in a preset ratio to form a composite modulation sequence.

[0038] The processing module is used to calculate the zero-order moment value, first-order moment value, and second-order central moment value corresponding to the current reconstruction frame segment from the time-series light intensity signal, determine the centroid coordinates of the moving target in the current reconstruction frame segment based on the zero-order moment value and first-order moment value after background compensation, and determine the rotation angle of the moving target in the current reconstruction frame segment based on the centroid coordinates and the second-order central moment value obtained by normalized difference processing.

[0039] The processing module is further configured to perform a reverse shift operation on the Hadamard modulation pattern of the current reconstructed frame segment in the composite modulation sequence using the motion offset parameter formed by the centroid coordinates and the rotation angle, to generate a motion-compensated reconstructed modulation pattern.

[0040] The execution module is used to perform correlation calculations between the temporal light intensity signal and the motion-compensated reconstructed modulation pattern to reconstruct a dynamic scene image sequence with motion blur eliminated.

[0041] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory storing code, the processor being configured to acquire the code and execute the above-described motion-compensated single-pixel real-time imaging method for dynamic scenes.

[0042] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described motion-compensated single-pixel real-time imaging method for dynamic scenes.

[0043] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0044] The motion-compensated single-pixel real-time imaging method and system for dynamic scenes provided in this application first projects a set of geometric moment modulation patterns, after differential normalization processing, onto a dynamic scene containing a moving target, and collects the temporal light intensity signal reflected back from the dynamic scene. The geometric moment modulation patterns and the Hadamard modulation patterns used for image reconstruction are interleaved at a preset ratio to form a composite modulation sequence. Second, the zero-order moment value, first-order moment value, and second-order central moment value corresponding to the current reconstruction frame segment are calculated from the temporal light intensity signal. Based on the zero-order moment value and first-order moment value after background compensation, the... The centroid coordinates of the moving target in the current reconstructed frame are determined, and the rotation angle of the moving target in the current reconstructed frame is determined based on the centroid coordinates and the second-order central moment value obtained by normalized difference processing. Then, the Hadamard modulation pattern of the current reconstructed frame in the composite modulation sequence is reverse-shifted using the motion offset parameter composed of the centroid coordinates and the rotation angle to generate a motion-compensated reconstructed modulation pattern. Finally, the temporal light intensity signal and the motion-compensated reconstructed modulation pattern are correlated to reconstruct a dynamic scene image sequence with motion blur eliminated.

[0045] Therefore, this application can avoid the problem of blurred reconstruction caused by the mismatch between the modulation pattern and the spatial position of the target due to non-cooperative moving targets in dynamic scenes. First, by interleaving the geometric moment modulation pattern and the Hadamard modulation pattern into a composite modulation sequence according to a preset ratio and projecting it onto the dynamic scene, motion parameter estimation and image reconstruction share the same set of temporal light intensity signals. This achieves synchronous multiplexing of motion perception and imaging functions within the limited sampling bandwidth of single-pixel imaging. At the same time, the geometric moment modulation pattern, after differential normalization processing, converts high dynamic range grayscale information into differential light intensity response, fundamentally avoiding the impact of quantization error on the moment value calculation accuracy caused by the binarization loading of grayscale patterns by the spatial light modulator. The adverse effects of the method are as follows: First, by synchronously calculating the zero-order moment, first-order moment, and second-order central moment from the time-series light intensity signal, and using the zero-order moment after background compensation to normalize the first-order moment to obtain the centroid coordinates, and using the centroid coordinates and the second-order central moment to construct the rotation invariant relationship to obtain the rotation angle, the translation and rotation of the moving target are independently calculated from the single measurement data within the same reconstruction frame segment. This allows for a quantitative description of rigid body planar motion without relying on inter-frame differencing or iterative optimization. Furthermore, the centroid coordinates and rotation angle strictly correspond to the target pose state at the same moment in both spatial and temporal dimensions, ensuring the spatiotemporal consistency of the motion offset parameters. Second, the centroid coordinates and rotation angle constitute a... The motion offset parameter is used to perform a reverse shift operation, including reverse rotation and reverse translation, on the Hadamard modulation pattern to generate a motion-compensated reconstructed modulation pattern. This operation pre-applies a geometric distortion to the measurement substrate that is opposite in direction and equal in amplitude to the actual motion of the target. This ensures that the modulation pattern maintains an equivalent spatial correspondence with the reference state in the target's body coordinate system. Therefore, the pattern misalignment effect caused by the target's motion is pre-canceled during subsequent correlation calculations. This avoids artifacts and resolution loss introduced by post-processing interpolation of the reconstructed image in motion compensation methods, as well as the mismatch in the spatiotemporal mapping between the Hadamard modulation pattern and the actual spatial distribution of the target caused by intra-frame motion drift of spatial position and orientation. Finally... The method involves performing inner product operations and inverse Hadamard transform on the Hadamard probe intensity value sequence extracted from the same set of time-series light intensity signals and the reconstructed modulation pattern after motion compensation to reconstruct a single-frame image. Since the modulation pattern has been pre-adapted to the current pose of the target, the contour and texture details of the moving target in the reconstructed image are not affected by the translation and rotation of the target within the frame segment. The single-frame images output sequentially by each consecutive reconstructed frame segment constitute a dynamic scene image sequence that eliminates motion blur, thereby achieving high-fidelity real-time observation of dynamic scenes within a single-pixel imaging framework. In summary, the technical solution provided in this application can avoid the correlation reconstruction blur problem caused by the mismatch between the modulation pattern and the target's spatial position due to non-cooperative moving targets in dynamic scenes. Attached Figure Description

[0046] Figure 1This is a schematic diagram of an application scenario architecture for a motion-compensated single-pixel real-time imaging method for dynamic scenes, as shown in some embodiments of this application.

[0047] Figure 2 This is an exemplary flowchart of a motion-compensated single-pixel real-time imaging method for dynamic scenes, according to some embodiments of this application.

[0048] Figure 3 This is an exemplary flowchart illustrating the determination of centroid coordinates according to some embodiments of this application;

[0049] Figure 4 This is a schematic diagram of the structure of a motion-compensated single-pixel real-time imaging system for dynamic scenes, as shown in some embodiments of this application.

[0050] Figure 5 This is a schematic diagram of the structure of a computer device that implements a motion-compensated single-pixel real-time imaging method for dynamic scenes, according to some embodiments of this application. Detailed Implementation

[0051] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] refer to Figure 1 This figure is a schematic diagram of an application scenario architecture for a motion-compensated single-pixel real-time imaging method for dynamic scenes, according to some embodiments of this application. The application scenario architecture includes a projection end, a dynamic moving target, a single-pixel detector, a computing unit, and the final imaging process. Specifically, the projection end first outputs a pre-arranged composite modulation sequence light field containing alternating geometric moment modulation patterns and Hadamard modulation patterns, which is projected onto the scene space where the dynamic moving target is located. After the dynamic moving target spatially modulates the light field, the reflected light signals carrying the target's motion and shape information are all converged onto the single image. The single-pixel detector completes the single-point acquisition of the entire time-series light intensity signal and transmits the data to the computing unit. The computing unit first calculates the zero-order moment, first-order moment and second-order central moment parameters of the corresponding frame segment from the time-series light intensity signal, and then solves for the motion offset parameters such as the real-time centroid coordinates and rotation angle of the moving target. Based on the motion offset parameters, the Hadamard modulation pattern is subjected to reverse shift motion compensation correction. Finally, through the matching correlation reconstruction operation between the light intensity signal and the compensated modulation pattern, the final imaging result with the elimination of motion artifacts and motion blur and the significant improvement in imaging clarity is output.

[0053] refer to Figure 2 This figure is an exemplary flowchart of a motion-compensated single-pixel real-time imaging method for dynamic scenes according to some embodiments of this application. The figure mainly includes the following steps:

[0054] In step S101, a set of geometric moment modulation patterns that have undergone differential normalization processing are projected onto a dynamic scene containing moving targets, and the temporal light intensity signal reflected back from the dynamic scene is acquired. The geometric moment modulation patterns and the Hadamard modulation patterns used for image reconstruction are interleaved in a preset ratio to form a composite modulation sequence.

[0055] In some embodiments, projecting a set of geometric moment modulation patterns, after differential normalization processing, onto a dynamic scene containing moving targets, and acquiring the temporal light intensity signal reflected back from the dynamic scene is achieved through the following steps:

[0056] A set of geometric moment modulation patterns is determined by projection after differential normalization, the geometric moment modulation patterns including a first-order moment modulation pattern after normalized differential processing and a second-order central moment modulation pattern after normalized differential processing;

[0057] The geometric moment modulation pattern is loaded onto a spatial light modulator, and the geometric moment modulation pattern is projected onto a dynamic scene containing moving targets via the spatial light modulator;

[0058] The time-series light intensity signal reflected back from the dynamic scene is acquired by a single-pixel detector.

[0059] In some embodiments, determining a set of geometric moment modulation patterns after differential normalization is achieved by the following steps:

[0060] A two-dimensional gray-level basis function matching the resolution of the spatial light modulator is generated, and the two-dimensional gray-level basis function is integrated in the corresponding order space to obtain a first-order gray-level moment basis pattern and a second-order gray-level central moment basis pattern.

[0061] The absolute values ​​of the positive and negative parts in the grayscale first-order moment base pattern are normalized to their maximum values ​​to obtain a first-order moment positive differential modulation sub-pattern and a first-order moment negative differential modulation sub-pattern, thereby obtaining a normalized differential first-order moment modulation pattern.

[0062] The absolute values ​​of the positive and negative parts in the grayscale second-order central moment base pattern are normalized to their maximum values ​​to obtain a second-order central moment positive differential modulation sub-pattern and a second-order central moment negative differential modulation sub-pattern, thereby obtaining a normalized differential second-order central moment modulation pattern.

[0063] A geometric moment modulation pattern is obtained by combining a first-order moment modulation pattern processed by normalized differential processing and a second-order central moment modulation pattern processed by normalized differential processing.

[0064] In specific implementation, firstly, a two-dimensional gray-level basis function matching the resolution of the spatial light modulator is generated. The resolution of the spatial light modulator is the number of micromirror units in the digital micromirror device array in the horizontal and vertical directions. The two-dimensional gray-level basis function is a continuous gray-level distribution function constructed in a continuous domain based on the mathematical definition of geometric moments, used to describe the spatial distribution characteristics of different order moment values. Integrating the two-dimensional gray-level basis function in the corresponding order space yields a first-order gray-level moment basis pattern and a second-order gray-level central moment basis pattern. The first-order gray-level moment basis pattern is a set of two-dimensional gray-level images in which the pixel gray-level values ​​change linearly with the pixel coordinates in the horizontal and vertical directions. The gray-level value of each pixel in its horizontal component pattern is equal to the pixel's horizontal coordinate multiplied by a preset ratio. The product of the components, the grayscale value of each pixel in the vertical component pattern is equal to the product of the pixel's ordinate and a preset scaling factor. The grayscale second-order central moment basis pattern is a set of two-dimensional grayscale images in which the pixel grayscale value changes quadratically with the square of the deviation of the pixel coordinate relative to the current centroid estimate. Its generation process is as follows: a local coordinate system is established with the current centroid estimate as the origin, and the square of the deviation of the horizontal coordinate and the square of the deviation of the vertical coordinate of each pixel in the local coordinate system are mapped to the corresponding grayscale value ranges respectively. Secondly, in order to avoid the influence of the quantization error caused by binarization processing when the spatial light modulator loads the high dynamic range grayscale pattern on the accuracy of subsequent moment value calculation, the absolute values ​​of the positive and negative parts in the grayscale first-order moment basis pattern are normalized to their maximum values ​​respectively. The maximum value normalization process involves dividing the gray value of each pixel in the positive part by the maximum gray value among all pixels in the positive part, so that the normalized gray values ​​of the positive part are distributed in the range of 0 to 1. Similarly, the gray value of each pixel in the absolute value of the negative part is divided by the maximum gray value among all pixels in the absolute value of the negative part, so that the normalized gray values ​​of the absolute value of the negative part are also distributed in the range of 0 to 1. After this processing, a first-order moment positive differential modulation sub-pattern and a first-order moment negative differential modulation sub-pattern are obtained. These two sub-patterns together constitute a normalized differentially processed first-order moment modulation pattern. This differential processing method allows the actual measured value of the first-order moment to be obtained through sequential... The positive and negative sub-patterns are projected and differential operations are performed on their corresponding detector light intensity values ​​to obtain the precise modulation pattern. During the differential operation, common-mode noise such as background light intensity and detector dark current is canceled out. Similarly, the absolute values ​​of the positive and negative parts in the grayscale second-order central moment base pattern are normalized to obtain a second-order central moment positive differential modulation sub-pattern and a second-order central moment negative differential modulation sub-pattern. The second-order central moment positive differential modulation sub-pattern and the second-order central moment negative differential modulation sub-pattern together constitute a normalized differentially processed second-order central moment modulation pattern. Finally, the above-mentioned normalized differentially processed first-order moment modulation pattern and normalized differentially processed second-order central moment modulation pattern are combined to obtain a geometric moment modulation pattern.

[0065] It should be noted that the geometric moment modulation pattern in this application is a set of all modulation patterns that need to be projected to complete the motion parameter calculation within a reconstruction frame. The geometric moment modulation pattern includes four differential modulation sub-patterns, namely, a first-order positive moment differential modulation sub-pattern, a first-order negative moment differential modulation sub-pattern, a second-order central moment positive differential modulation sub-pattern, and a second-order central moment negative differential modulation sub-pattern. After these four sub-patterns are loaded onto the spatial light modulator in a preset order, four sets of corresponding detection light intensity values ​​can be collected by a single-pixel detector. The centroid coordinates and rotation angle of the moving target in the current reconstruction frame are calculated by differential operation and ratio operation from these four sets of detection light intensity values.

[0066] Specifically, the first-order moment modulation pattern and the second-order central moment modulation pattern, which have undergone normalized differential processing, are sequentially loaded into the spatial light modulator. This means that the first-order positive differential modulation sub-pattern, the first-order negative differential modulation sub-pattern, the second-order positive central moment differential modulation sub-pattern, and the second-order negative central moment differential modulation sub-pattern are sequentially loaded into the spatial light modulator. The spatial light modulator is a digital micromirror device (DMD), which modulates the spatial intensity of the incident beam by controlling the deflection state of each micromirror unit. The geometric moment modulation pattern is then projected onto a dynamic scene containing a moving target. The dynamic scene consists of one or more objects in motion and their static background environment. During projection, the first-order moment modulation pattern and the second-order central moment modulation pattern, which have undergone normalized differential processing, are applied to the spatial light modulator frame by frame in a preset time sequence. The beam emitted by the light source, after being reflected by the spatial light modulator, carries the spatial structure information of the modulation pattern and illuminates the surface of the dynamic scene.

[0067] Specifically, a single-pixel detector is used to collect the temporal light intensity signal reflected back from the dynamic scene. The single-pixel detector is a photodetector that does not have spatial resolution but has high temporal resolution. It converts the total light flux received after reflection or transmission from the dynamic scene into a time-corresponding electrical signal sequence. Specifically, it sequentially receives the first positive light intensity value reflected back from the dynamic scene when the first-order moment positive differential modulation sub-pattern of the geometric moment modulation pattern is projected, the first negative light intensity value reflected back from the dynamic scene when the first-order moment negative differential modulation sub-pattern of the geometric moment modulation pattern is projected, the second positive light intensity value reflected back from the dynamic scene when the second-order central moment positive differential modulation sub-pattern of the geometric moment modulation pattern is projected, and the second negative light intensity value reflected back from the dynamic scene when the second-order central moment negative differential modulation sub-pattern of the geometric moment modulation pattern is projected. The above light intensity values ​​together constitute the part of the temporal light intensity signal corresponding to the geometric moment modulation pattern, and this temporal light intensity signal will serve as the common data basis for subsequent motion parameter calculation and image reconstruction.

[0068] It should be noted that, in this application, the temporal intensity signal refers to a set of one-dimensional voltage response sequences continuously acquired and recorded in chronological order by a single-pixel detector during the process of the spatial light modulator projecting modulation patterns frame by frame. The voltage amplitude corresponding to each time sampling point characterizes the total luminous flux intensity generated on the photosensitive surface of the detector after the dynamic scene reflects or transmits the spatial light field distribution of the currently projected modulation pattern as a whole. The temporal order of this temporal intensity signal corresponds strictly synchronously with the projection order of each frame of modulation pattern in the composite modulation sequence, providing a temporal index basis for subsequently separating and extracting the detection light intensity value corresponding to the geometric moment modulation pattern and the detection light intensity value corresponding to the Hadamard modulation pattern from this signal sequence according to different modulation pattern types.

[0069] In some embodiments, the geometric moment modulation pattern and the Hadamard modulation pattern used for image reconstruction are interleaved at a preset ratio to form a composite modulation sequence, which is achieved by the following steps:

[0070] The total number of Hadamard modulation patterns and the total number of geometric moment modulation patterns required to be included in the current reconstructed frame segment are determined according to the preset composite interleaving ratio.

[0071] A corresponding number of Hadamard base patterns are generated according to the total number of Hadamard modulation patterns, and the Hadamard base patterns are sorted in order of energy concentration from high to low to obtain a Hadamard sorting sequence.

[0072] The geometric moment modulation pattern is used as a geometric moment measurement unit, and the geometric moment measurement unit is inserted into the Hadamard sorting sequence at equal intervals. Then, the Hadamard sorting sequence after inserting the geometric moment measurement unit is determined as the composite modulation sequence.

[0073] In specific implementation, firstly, the total number of Hadamard modulation patterns and the total number of geometric moment modulation patterns to be included in the current reconstruction frame segment are determined according to a preset composite interleaving ratio. The preset composite interleaving ratio is a dimensionless parameter used to characterize the proportion of the geometric moment measurement pattern in all modulation patterns. Its value is determined based on the average motion speed of the moving target in the dynamic scene and the maximum allowable single-frame reconstruction delay of the system (i.e., the average motion speed of the moving target in the imaging plane is obtained by differentiating the centroid coordinate sequence calculated in the previous reconstruction frame segment, and the single-frame delay is determined by the average motion speed, the system light source modulation rate, and the detector sampling bandwidth). Multiplying the maximum reconstruction delay yields the maximum possible displacement of the target within a single reconstruction frame. This maximum possible displacement is then divided by a preset tolerance threshold and rounded up to obtain the minimum number of geometric moment measurement units (MMMUs) to be inserted within a reconstruction frame. The reciprocal of this minimum number is multiplied by the number of four sub-pattern frames contained in the MMU. The result is used as the lower limit of the composite interleaving ratio (the actual value is selected between this lower limit and the upper limit determined by the minimum Hadamard sampling rate constraint). The total number of Hadamard modulation patterns is the total number of frames of Hadamard base patterns sequentially projected for image reconstruction within a reconstruction frame. The geometric moment modulation patterns... The total number refers to the total number of frames of geometric moment modulation patterns projected sequentially within the same reconstructed frame segment for motion parameter calculation. In a typical configuration, the total number of geometric moment modulation patterns is fixed at four frames, corresponding to the first-order moment positive differential modulation sub-pattern, the first-order moment negative differential modulation sub-pattern, the second-order central moment positive differential modulation sub-pattern, and the second-order central moment negative differential modulation sub-pattern, respectively. Next, a corresponding number of Hadamard base patterns are generated according to the total number of Hadamard modulation patterns. The Hadamard base pattern is a binary spatial modulation pattern formed by two-dimensional rearrangement of the row vectors or column vectors of the Hadamard matrix. Each frame of the Hadamard base pattern corresponds to a specific element in the Hadamard matrix. The Hadamard pattern in each frame, whether in rows or columns, is spatially represented as an orthogonal stripe structure with alternating bright and dark areas. The Hadamard patterns are sorted in descending order of energy concentration to obtain a Hadamard sorting sequence. The energy concentration is the proportion of the total energy of the low-frequency component in the frequency domain of the Hadamard pattern. The sorting operation is achieved by calculating the number of zero crossings of each Hadamard pattern. In the sorted Hadamard sequence, the pattern at the beginning corresponds to the lower spatial frequency component, and the pattern at the end corresponds to the higher spatial frequency component. This sorting strategy allows for the priority acquisition of low-frequency information that contributes the most to the image reconstruction quality under undersampling conditions.Finally, the geometric moment modulation pattern is used as a geometric moment measurement unit. This geometric moment measurement unit is an indivisible set of patterns containing complete first-order moment positive differential modulation sub-patterns, first-order moment negative differential modulation sub-patterns, second-order central moment positive differential modulation sub-patterns, and second-order central moment negative differential modulation sub-patterns. The four frames of sub-patterns in this set are projected sequentially as a whole to ensure that each moment value in the motion parameter calculation corresponds to the target pose state at the same time. The geometric moment measurement unit is inserted into the Hadamard sorting sequence at equal intervals. The equal intervals refer to the Hadamard base pattern in the Hadamard sorting sequence at fixed intervals of a fixed number of frames. A complete set of geometric moment measurement units is inserted, with the interval step size determined by the ratio of the composite interleaving ratio to the total number of Hadamard modulation patterns. The insertion operation is completed at the sequence index level without changing the pixel data of each pattern itself. The Hadamard sorting sequence after inserting the geometric moment measurement units is determined as the composite modulation sequence. Within each reconstructed frame segment, the four differential modulation sub-patterns contained in the geometric moment measurement unit are forcibly arranged and projected before all Hadamard modulation patterns to ensure that the centroid coordinates and rotation angles calculated from this geometric moment measurement unit can be used to compensate for all subsequent Hadamard modulation patterns within the same frame segment, avoiding motion parameter prediction failures caused by timing misalignment.

[0074] It should be noted that the composite modulation sequence in this application is a complete sequence of all modulation patterns in a reconstructed frame arranged in the order of projection time. This sequence is stored in the frame buffer of the control unit in the form of a data structure that can be loaded and projected frame by frame by the spatial light modulator. Subsequently, it will be applied to the spatial light modulator frame by frame to complete the structured light illumination and single pixel detection of the dynamic scene.

[0075] In step S102, the zero-order moment, first-order moment, and second-order central moment corresponding to the current reconstructed frame are calculated from the temporal light intensity signal. The centroid coordinates of the moving target in the current reconstructed frame are determined based on the zero-order moment and first-order moment after background compensation. The rotation angle of the moving target in the current reconstructed frame is determined based on the centroid coordinates and the second-order central moment obtained by normalized difference processing.

[0076] In some embodiments, the zeroth moment, first moment, and second central moment corresponding to the current reconstructed frame segment are calculated from the temporal light intensity signal using the following steps:

[0077] Extract four probe light intensity values ​​corresponding to the four differential modulation sub-patterns contained in the geometric moment measurement unit within the current reconstructed frame segment from the time-series light intensity signal. The four probe light intensity values ​​are composed of a first positive light intensity value, a first negative light intensity value, a second positive light intensity value, and a second negative light intensity value.

[0078] Perform a difference operation between the first positive light intensity value and the first negative light intensity value to obtain a first-order moment difference detection value, and determine the first-order moment difference detection value as the first-order moment value corresponding to the current reconstructed frame segment;

[0079] Perform a difference operation between the second positive light intensity value and the second negative light intensity value to obtain the second-order central moment difference detection value, and determine the second-order central moment difference detection value as the second-order central moment value corresponding to the current reconstructed frame segment;

[0080] Extract the light intensity detection value corresponding to the fully bright pattern in the Hadamard modulation pattern within the current reconstructed frame segment, and determine the light intensity detection value corresponding to the fully bright pattern as the zero-order moment value corresponding to the current reconstructed frame segment.

[0081] Specifically, when generating the Hadamard modulation pattern, the eigenvectors whose elements in the first row or first column of the Hadamard matrix are all positive are rearranged in two dimensions to form a fully bright pattern. This fully bright pattern is the modulation pattern corresponding to all micromirror units on the spatial light modulator being in the on state. Physically, it is equivalent to reflecting all the luminous flux emitted by the light source to the dynamic scene without applying any spatial modulation to the incident light field. Since the Hadamard modulation pattern and the geometric moment modulation pattern in the single-pixel imaging system are projected sequentially according to a preset composite interleaved arrangement order, the temporal position of the fully bright pattern within the current reconstructed frame is uniquely determined by this composite interleaved arrangement order. Based on the corresponding temporal position index, the projection time of the fully bright pattern is extracted from the continuously acquired temporal light intensity signal. The voltage response amplitude is the light intensity detection value corresponding to the fully bright pattern. According to the mathematical definition of geometric moments, the zeroth moment represents the total light intensity integral of the target scene in the imaging plane. For a single-pixel imaging system modulated by a spatial light modulator, when a fully bright pattern is projected, the total light flux received by the single-pixel detector is the sum of all reflected light intensities of the current dynamic scene under conditions without spatial modulation. Therefore, the light intensity detection value corresponding to the fully bright pattern is completely equivalent to the zeroth moment in physical meaning. In actual operation, the extracted light intensity detection value corresponding to the fully bright pattern is directly assigned to the register in the variable storage unit used to record the zeroth moment value, thus completing the operation of determining the light intensity detection value corresponding to the fully bright pattern as the zeroth moment value corresponding to the current reconstructed frame segment.

[0082] It should be noted that, in this application, the zeroth-order moment value is used to characterize the total reflected light flux of the moving target in the imaging plane, and the zeroth-order moment value is the original zeroth-order moment value, which includes the static background reflected light intensity. In this application, the first-order moment value is used to characterize the first-order spatial weighted integral of the moving target's reflected light intensity distribution along the corresponding coordinate axis direction, and the second-order central moment value is used to characterize the second-order spatial dispersion of the moving target's reflected light intensity distribution relative to the current centroid coordinates. All three moment values ​​are obtained within the same reconstruction frame segment through time-division projection and synchronous acquisition of the composite modulation sequence, and together they constitute a complete moment description of the moving target's pose parameters within the reconstruction frame segment.

[0083] In some embodiments, reference Figure 3 As shown, this figure is an exemplary flowchart of determining the centroid coordinates according to some embodiments of this application. In this embodiment, determining the centroid coordinates of the moving target in the current reconstructed frame segment based on the zeroth-order moment value and the first-order moment value after background compensation can be achieved by the following steps:

[0084] In step S1021, the zero-order moment value of the static background obtained by pre-calibration is subtracted from the zero-order moment value corresponding to the current reconstructed frame segment to obtain the zero-order moment value after background compensation.

[0085] In step S1022, the first moment value corresponding to the current reconstructed frame segment is compared with the zero moment value after background compensation by a dimension-wise ratio operation to obtain the centroid abscissa and centroid ordinate of the moving target in the current reconstructed frame segment.

[0086] In step S1023, the centroid coordinates of the moving target in the current reconstructed frame are formed by the centroid abscissa and the centroid ordinate together.

[0087] In specific implementation, firstly, during the initialization phase of the single-pixel imaging system, a full-brightness pattern projection and detection is performed on a static background scene without moving targets. The single-pixel detector collects the total reflected light intensity value of the static background scene under conditions without spatial modulation, and uses this total reflected light intensity value as the static background zero-order moment value. The static background zero-order moment value represents the constant light intensity bias contributed by all stationary background objects and environmental stray light within the imaging field of view. During the real-time processing of each reconstructed frame segment, the pre-calibrated static background zero-order moment value is subtracted from the zero-order moment value corresponding to the current reconstructed frame segment. This subtraction operation is achieved by performing binary two's complement subtraction between the digital signal corresponding to the zero-order moment value of the current reconstructed frame segment and the static background zero-order moment value within the arithmetic unit. The resulting difference is the zero-order moment value after background compensation. The zero-order moment value after background compensation refers to the reflection of the light flux reflected by the moving target itself in the imaging after removing background light intensity and detector dark current interference. The total integral value in the plane is calculated. Then, the first-order moment value corresponding to the current reconstructed frame is compared with the zero-order moment value after background compensation in a dimension-wise ratio operation. The dimension-wise ratio operation means dividing the horizontal first-order moment component corresponding to the horizontal direction of the first-order moment value by the zero-order moment value after background compensation to obtain the centroid horizontal coordinate, and simultaneously dividing the vertical first-order moment component corresponding to the vertical direction of the first-order moment value by the same zero-order moment value after background compensation to obtain the centroid vertical coordinate. This operation is based on the mathematical relationship in geometric moment theory that the quotient of the first-order moment and the zero-order moment is the centroid position of the target area. The horizontal and vertical components of the first-order moment value are provided by the first-order moment difference detection values ​​in the horizontal and vertical directions, respectively. The zero-order moment value after background compensation is used as a divisor on the two components to ensure the dimensionlessness and brightness invariance of the centroid coordinates. Finally, the centroid horizontal coordinate and the centroid vertical coordinate together constitute the centroid coordinates of the moving target in the current reconstructed frame.

[0088] It should be noted that, in this application, the centroid coordinates refer to a two-dimensional ordered pair determined by the dimension-wise ratio operation between the first-order moment value calculated within the current reconstructed frame and the zero-order moment value after background compensation. In the pixel coordinate system of the spatial light modulator, the centroid coordinates characterize the geometric center position of the reflected light intensity distribution of the moving target in the imaging plane. That is, under the assumption that the target light intensity distribution is uniform or centrally symmetric, the equilibrium point coordinates of the weighted integral of the target's light intensity in all directions are obtained. As a quantitative description parameter of the target's translational motion, the centroid coordinates directly provide the rotation center reference position for the subsequent rotation angle calculation, and at the same time provide the calculation basis for the translational reverse component of the reverse shift compensation operation of the Hadamard modulation pattern.

[0089] In some embodiments, determining the rotation angle of the moving target in the current reconstructed frame segment based on the centroid coordinates and the second-order central moment value obtained through normalized difference processing is achieved through the following steps:

[0090] A rotation invariant relation for the second-order central moment is constructed with the centroid coordinates as the rotation center. The second-order central moment value corresponding to the current reconstructed frame segment is substituted into the rotation invariant relation to solve for the change in the principal axis azimuth angle of the moving target relative to the reference frame segment in the current reconstructed frame segment.

[0091] The change in the principal axis azimuth angle is determined as the rotation angle of the moving target in the current reconstructed frame.

[0092] In specific implementation, firstly, a rotational invariant relation for the second-order central moments is constructed using the centroid coordinates as the rotation center. This rotational invariant relation is an equality constraint established based on the mathematical properties of the second-order central moments under rigid body rotation transformations in image moment theory. The specific construction process is as follows: the horizontal, vertical, and mixed second-order central moment components of the second-order central moment value corresponding to the current reconstructed frame are combined to form a second-order central moment matrix. This second-order central moment matrix is ​​a symmetric positive definite matrix, with its main diagonal elements equal to the horizontal and vertical second-order central moment components, and its secondary diagonal elements equal to the mixed second-order central moment component. The eigenvalues ​​of this second-order central moment matrix characterize the dispersion of the moving target's light intensity distribution along the principal axis and perpendicular to the principal axis. The eigenvectors of this second-order central moment matrix characterize the orientation of the principal axis of the moving target's light intensity distribution within the imaging plane. Subsequently, the second-order central moment value corresponding to the current reconstructed frame is substituted into... The rotation invariant relationship is used to solve for the change in principal axis azimuth angle of the moving target in the current reconstructed frame segment relative to the reference frame segment. The reference frame segment is a specific reconstructed frame segment specified by the user, and its corresponding second-order central moment matrix and principal axis azimuth angle are pre-stored in the reference parameter register. The solution operation is achieved by calculating the rotation transformation relationship between the second-order central moment matrix of the current reconstructed frame segment and the second-order central moment matrix of the reference frame segment. Specifically, using the transformation property of the second-order central moment matrix under rigid body rotation, the tangent double angle expression of the change in principal axis azimuth angle is constructed from the algebraic combination relationship between the elements of the current second-order central moment matrix and the reference second-order central moment matrix. The arctangent operation is performed on the tangent double angle expression, and the quadrant correction is determined according to the sign of the mixed second-order central moment components, that is, the change in principal axis azimuth angle in terms of angle is obtained. Finally, the change in principal axis azimuth angle is determined as the rotation angle of the moving target in the current reconstructed frame segment.

[0093] It should be noted that the rotation angle in this application is a scalar angle value describing the clockwise or counterclockwise rotation of the moving target around its own center of mass in the imaging plane. This rotation angle will be used as the rotation component of the motion offset parameter and together with the translation component of the center of mass coordinates to form a complete motion offset parameter, providing an angle compensation basis for subsequent reverse shift operations including rotation inversion transformation and translation inversion transformation on the Hadamard modulation pattern.

[0094] In step S103, the Hadamard modulation pattern of the current reconstructed frame segment in the composite modulation sequence is reverse-shifted using the motion offset parameter formed by the centroid coordinates and the rotation angle to generate the motion-compensated reconstructed modulation pattern.

[0095] In some embodiments, the following steps are used to perform a reverse shift operation on the Hadamard modulation pattern of the current reconstructed frame segment in the composite modulation sequence using the motion offset parameter formed by the centroid coordinates and the rotation angle to generate the motion-compensated reconstructed modulation pattern:

[0096] The translational reverse component is formed by the opposite of the centroid coordinates, and the rotational reverse component is formed by the opposite of the rotation angle. The translational reverse component and the rotational reverse component are combined to form the reverse compensation parameter corresponding to the motion offset parameter.

[0097] Based on the inverse compensation parameters, the inverse rotation transformation and inverse translation transformation are sequentially performed on the coordinates of each pixel in the Hadamard modulation pattern corresponding to the current reconstructed frame segment to obtain the corresponding coordinate position of each pixel in the compensated modulation pattern.

[0098] The original grayscale value of each pixel is assigned to its corresponding coordinate position after inverse rotation and inverse translation transformation, thereby generating a motion-compensated reconstructed modulation pattern.

[0099] In specific implementation, firstly, a translational reverse component is constructed using the negative value of the centroid coordinates, and a rotational reverse component is constructed using the negative value of the rotation angle. The translational reverse component and the rotational reverse component are then combined to form the reverse compensation parameter corresponding to the motion offset parameter. The negative value of the centroid coordinates is a two-dimensional translation vector obtained by taking the negative values ​​of the centroid's horizontal and vertical coordinates. This two-dimensional translation vector has the same amplitude as the actual translational displacement of the moving target from the reference frame to the current reconstructed frame, but in the opposite direction. The negative value of the rotation angle is a scalar obtained by taking the negative value of the principal axis azimuth change. An angle value, equal in magnitude but opposite in direction to the actual rotation angle of the moving target around its centroid from the reference frame to the current reconstructed frame, is used. The reverse compensation parameter is a set of composite geometric transformation parameters including translation and rotation reverse components. Next, based on the reverse compensation parameter, reverse rotation and reverse translation transformations are sequentially performed on the pixel coordinates of each pixel in the Hadamard modulation pattern corresponding to the current reconstructed frame to obtain the corresponding coordinate positions of each pixel in the compensated modulation pattern. The Hadamard modulation pattern is a set of Hadamard values ​​extracted from the composite modulation sequence according to the reconstructed frame for image reconstruction. The base pattern, each frame of the Hadamard modulation pattern, is composed of a two-dimensional pixel array, and each pixel has a defined original pixel coordinate and original grayscale value. The reverse rotation transformation involves rotating the original pixel coordinates of each pixel in the Hadamard modulation pattern around an axis perpendicular to the imaging plane by an angle value given by the rotation reverse component, with the centroid coordinate as the rotation center. The transformation process is achieved through rotation matrix multiplication. Let the offset vector of a pixel relative to the centroid coordinate be the difference coordinate. Multiplying the difference coordinate by the rotation matrix formed by combining the cosine and sine of the rotation angle, and then adding it back to the centroid coordinate, yields the intermediate pixel after reverse rotation. The inverse translation transformation is to superimpose the inverse translation component onto the intermediate pixel coordinates obtained by the inverse rotation transformation. That is, the horizontal component value of the inverse translation component is added to the horizontal coordinate of the intermediate pixel coordinate, and the vertical component value of the inverse translation component is added to the vertical coordinate of the intermediate pixel coordinate to obtain the corresponding coordinate position of the pixel in the compensated modulation pattern. The above transformation order of rotation followed by translation strictly conforms to the kinematic decomposition relationship of rigid body kinematics when the rotation center is located at the center of mass. If the obtained corresponding coordinate position exceeds the pixel boundary of the original Hadamard modulation pattern during the transformation process, the coordinate position is marked as an invalid position.Finally, the original grayscale value of each pixel is assigned to its corresponding coordinate position after inverse rotation and inverse translation transformations to generate the motion-compensated reconstructed modulation pattern. The original grayscale value is the grayscale modulation value of each pixel in the Hadamard modulation pattern before any transformation is performed. The assignment operation is achieved by creating a blank image buffer with the same resolution as the original Hadamard modulation pattern and writing the pixel grayscale value at the effective corresponding coordinate position into the buffer. For cases where multiple original pixels are mapped to the same corresponding coordinate position after transformation, an averaging strategy is used to determine the grayscale value of the corresponding coordinate position. For corresponding coordinate positions without any original pixel mapping after transformation, a nearest neighbor interpolation strategy is used to determine the grayscale value of the corresponding coordinate position. After all Hadamard modulation patterns have been processed by the above inverse shift operation, a set of motion-compensated reconstructed modulation patterns corresponding one-to-one with the original Hadamard modulation patterns is obtained.

[0100] It should be noted that the motion-compensated reconstructed modulation pattern in this application refers to a set of new modulation patterns generated after performing an inverse geometric transformation on the original Hadamard modulation pattern used for image reconstruction in the current reconstruction frame. This set of new modulation patterns has the same orthogonality and completeness as the original Hadamard modulation pattern in spatial structure, but the pixel coordinate distribution has been artificially pre-distorted. The direction of the distortion is opposite to the direction of the translation and rotation of the moving target in the current reconstruction frame and the amplitude is equal. This makes it possible that when this set of new modulation patterns is projected onto the surface of the moving target that has undergone pose change, the spatial modulation code received by each local area of ​​the target surface is equivalent to the spatial modulation code received by the target in the static state of the reference frame in the target's own body coordinate system. As a result, when the Hadamard detection light intensity value sequence synchronously collected by the single pixel detector is subsequently correlated with the motion-compensated reconstructed modulation pattern, the misalignment effect between the modulation pattern and the target surface caused by the target motion is pre-canceled, and the motion blur in the reconstructed single-frame image is eliminated.

[0101] In step S104, the temporal light intensity signal and the reconstructed modulation pattern after motion compensation are correlated and calculated to reconstruct a dynamic scene image sequence with motion blur eliminated.

[0102] In some embodiments, the correlation calculation between the temporal light intensity signal and the motion-compensated reconstructed modulation pattern to reconstruct a dynamic scene image sequence without motion blur is achieved through the following steps:

[0103] Extract the sequence of Hadamard probe light intensity values ​​corresponding to the Hadamard modulation patterns of each frame in the current reconstructed frame segment from the time-series light intensity signal;

[0104] Perform an inner product operation between each probe light intensity value in the sequence of Hadamard probe light intensity values ​​and the modulation pattern of the corresponding frame in the motion-compensated reconstructed modulation pattern to obtain the one-dimensional Hadamard transform coefficient vector corresponding to the current reconstructed frame segment.

[0105] Perform an inverse Hadamard transform on the one-dimensional Hadamard transform coefficient vector to reconstruct a single frame image corresponding to the reconstructed frame segment in the dynamic scene image sequence with motion blur eliminated.

[0106] In specific implementation, firstly, a sequence of Hadamard probe light intensity values ​​corresponding one-to-one with the Hadamard modulation patterns of each frame in the current reconstructed frame segment is extracted from the time-series light intensity signal. The time-series light intensity signal is a one-dimensional voltage response sequence continuously acquired by a single-pixel detector during frame-by-frame projection of the composite modulation sequence. Each sampling point is strictly synchronized with the projection timing of the modulation pattern. The extraction operation of the Hadamard probe light intensity value sequence is based on the temporal position index of the Hadamard modulation pattern in the composite modulation sequence determined by the composite interleaved arrangement order. The corresponding voltage amplitude is extracted point by point from the time-series light intensity signal according to the index. Each extracted value is arranged sequentially according to the projection order of its corresponding Hadamard modulation pattern to form a one-dimensional vector. This vector is the Hadamard probe light intensity sequence. The sequence of Hadamard probe light intensity values, where each element represents the total intensity of the light field response of the dynamic scene to a specific Hadamard substrate pattern in a given frame within the current reconstructed frame. Next, each probe light intensity value in the Hadamard probe light intensity value sequence is multiplied by the modulation pattern of the corresponding frame in the motion-compensated reconstructed modulation pattern to obtain the one-dimensional Hadamard transform coefficient vector corresponding to the current reconstructed frame. The inner product operation is defined as an algebraic operation that multiplies the k-th scalar value in the Hadamard probe light intensity value sequence with the k-th frame's two-dimensional modulation pattern in the motion-compensated reconstructed modulation pattern set pixel-by-pixel after unfolding into a one-dimensional vector, and then sums the results. This is specifically executed by the multiply-accumulate unit in the digital signal processor. The process is as follows: Each frame of the reconstructed modulation pattern set after motion compensation is expanded into a one-dimensional vector in row-major or column-major order. The number of elements in this one-dimensional vector equals the total number of pixels in the spatial light modulator. This one-dimensional vector is multiplied by the corresponding scalar value of the Hadamard probe light intensity. The resulting product vector is used as the weighted coefficient contribution vector. The weighted coefficient contribution vectors corresponding to all frames are summed element-wise. The final one-dimensional vector with the same length as the total number of pixels in the spatial light modulator is the one-dimensional Hadamard transform coefficient vector. Each element in the one-dimensional Hadamard transform coefficient vector corresponds to the expansion coefficient of the target scene on the Hadamard orthogonal basis. Since the modulation pattern participating in the inner product operation has undergone reverse shift compensation, this expansion... The coefficients reflect the spatial frequency component distribution of the target after eliminating the influence of its rigid body motion. Finally, the inverse Hadamard transform is performed on the one-dimensional Hadamard transform coefficient vector to reconstruct the single-frame image corresponding to the reconstructed frame segment in the dynamic scene image sequence with motion blur eliminated. The inverse Hadamard transform is the inverse operation of the forward Hadamard transform. Specifically, the Hadamard transform coefficient vector is multiplied by the inverse Hadamard transform matrix. Since the Hadamard matrix is ​​a symmetric orthogonal matrix, its inverse Hadamard transform matrix is ​​equal to itself divided by the matrix order. Therefore, in engineering implementation, the inverse Hadamard transform is usually completed by the fast Hadamard transform algorithm to reconstruct the single-frame image corresponding to the reconstructed frame segment in the dynamic scene image sequence with motion blur eliminated.

[0107] It should be noted that the single-frame image of the reconstructed frame segment in this application refers to the single-frame image with motion blur of the moving target eliminated.

[0108] Furthermore, in another aspect of this application, in some embodiments, this application provides a motion-compensated single-pixel real-time imaging system for dynamic scenes, with reference to... Figure 4 The figure is a schematic diagram of the structure of a motion-compensated single-pixel real-time imaging system for dynamic scenes according to some embodiments of this application. The motion-compensated single-pixel real-time imaging system for dynamic scenes includes: an acquisition module 201, a processing module 202, and an execution module 203, which are described below:

[0109] Acquisition module 201, in this application, is mainly used to project a set of geometric moment modulation patterns that have undergone differential normalization processing onto a dynamic scene containing moving targets, and to acquire the temporal light intensity signal reflected back from the dynamic scene. The geometric moment modulation patterns and the Hadamard modulation patterns used for image reconstruction are interleaved in a preset ratio to form a composite modulation sequence.

[0110] Processing module 202 in this application is mainly used to calculate the zero-order moment value, first-order moment value and second-order central moment value corresponding to the current reconstruction frame segment from the time-series light intensity signal, determine the centroid coordinates of the moving target in the current reconstruction frame segment based on the zero-order moment value and first-order moment value after background compensation, and determine the rotation angle of the moving target in the current reconstruction frame segment based on the centroid coordinates and the second-order central moment value obtained by normalized difference processing.

[0111] The processing module 202 is further configured to perform a reverse shift operation on the Hadamard modulation pattern of the current reconstructed frame segment in the composite modulation sequence using the motion offset parameter formed by the centroid coordinates and the rotation angle, to generate a motion-compensated reconstructed modulation pattern.

[0112] The execution module 203 in this application is mainly used to perform correlation calculations on the temporal light intensity signal and the reconstructed modulation pattern after motion compensation, and reconstruct a dynamic scene image sequence with motion blur eliminated.

[0113] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, the processor being configured to acquire the code and execute the above-described motion-compensated single-pixel real-time imaging method for dynamic scenes.

[0114] In some embodiments, reference Figure 5The figure is a schematic diagram of the structure of a computer device implementing a motion-compensated single-pixel real-time imaging method for dynamic scenes, according to some embodiments of this application. The motion-compensated single-pixel real-time imaging method for dynamic scenes in the above embodiments can... Figure 5 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0115] The processor 301 can be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more devices used to control the execution of the motion-compensated single-pixel real-time imaging method for dynamic scenes in this application.

[0116] The communication bus 302 can be used to transmit information between the aforementioned components.

[0117] The memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0118] The memory 303 stores program code for executing the scheme of this application, and its execution is controlled by the processor 301. The processor 301 executes the program code stored in the memory 303. The program code may include one or more software modules. In the above embodiments, the determination of the motion-compensated single-pixel real-time imaging method for dynamic scenes can be implemented by the processor 301 and one or more software modules in the program code in the memory 303.

[0119] Communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0120] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0121] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0122] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described motion-compensated single-pixel real-time imaging method for dynamic scenes.

[0123] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0124] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A motion-compensated single-pixel real-time imaging method for dynamic scenes, characterized in that, Includes the following steps: A set of geometric moment modulation patterns, after differential normalization processing, is projected onto a dynamic scene containing moving targets, and the temporal light intensity signal reflected back from the dynamic scene is collected. The geometric moment modulation patterns and the Hadamard modulation patterns used for image reconstruction are interleaved in a preset ratio to form a composite modulation sequence. The zero-order moment, first-order moment, and second-order central moment corresponding to the current reconstructed frame are calculated from the time-series light intensity signal. The centroid coordinates of the moving target in the current reconstructed frame are determined based on the zero-order moment and first-order moment after background compensation. The rotation angle of the moving target in the current reconstructed frame is determined based on the centroid coordinates and the second-order central moment obtained by normalized difference processing. The Hadamard modulation pattern of the current reconstructed frame segment in the composite modulation sequence is reverse-shifted using the motion offset parameter formed by the centroid coordinates and the rotation angle to generate a motion-compensated reconstructed modulation pattern. The temporal light intensity signal and the motion-compensated reconstructed modulation pattern are correlated and calculated to reconstruct a dynamic scene image sequence with motion blur eliminated.

2. The method as described in claim 1, characterized in that, Projecting a set of geometric moment modulation patterns, processed by differential normalization, onto a dynamic scene containing moving targets, and acquiring the temporal light intensity signal reflected back from the dynamic scene specifically includes: A set of geometric moment modulation patterns is determined by projection after differential normalization, the geometric moment modulation patterns including a first-order moment modulation pattern after normalized differential processing and a second-order central moment modulation pattern after normalized differential processing; The geometric moment modulation pattern is loaded onto a spatial light modulator, and the geometric moment modulation pattern is projected onto a dynamic scene containing moving targets via the spatial light modulator; The time-series light intensity signal reflected back from the dynamic scene is acquired by a single-pixel detector.

3. The method as described in claim 2, characterized in that, The specific components of the set of geometric moment modulation patterns determined by the differential normalization process include: A two-dimensional gray-level basis function matching the resolution of the spatial light modulator is generated, and the two-dimensional gray-level basis function is integrated in the corresponding order space to obtain a first-order gray-level moment basis pattern and a second-order gray-level central moment basis pattern. The absolute values ​​of the positive and negative parts in the grayscale first-order moment base pattern are normalized to their maximum values ​​to obtain a first-order moment positive differential modulation sub-pattern and a first-order moment negative differential modulation sub-pattern, thereby obtaining a normalized differential first-order moment modulation pattern. The absolute values ​​of the positive and negative parts in the grayscale second-order central moment base pattern are normalized to their maximum values ​​to obtain a second-order central moment positive differential modulation sub-pattern and a second-order central moment negative differential modulation sub-pattern, thereby obtaining a normalized differential second-order central moment modulation pattern. A geometric moment modulation pattern is obtained by combining a first-order moment modulation pattern processed by normalized differential processing and a second-order central moment modulation pattern processed by normalized differential processing.

4. The method as described in claim 1, characterized in that, The geometric moment modulation pattern and the Hadamard modulation pattern used for image reconstruction are interleaved at a preset ratio to form a composite modulation sequence, specifically including: The total number of Hadamard modulation patterns and the total number of geometric moment modulation patterns required to be included in the current reconstructed frame segment are determined according to the preset composite interleaving ratio. A corresponding number of Hadamard base patterns are generated according to the total number of Hadamard modulation patterns, and the Hadamard base patterns are sorted in order of energy concentration from high to low to obtain a Hadamard sorting sequence. The geometric moment modulation pattern is used as a geometric moment measurement unit, and the geometric moment measurement unit is inserted into the Hadamard sorting sequence at equal intervals. Then, the Hadamard sorting sequence after inserting the geometric moment measurement unit is determined as the composite modulation sequence.

5. The method as described in claim 1, characterized in that, The calculation of the zeroth moment, first moment, and second central moment corresponding to the current reconstructed frame segment from the time-series light intensity signal specifically includes: Extract four probe light intensity values ​​corresponding to the four differential modulation sub-patterns contained in the geometric moment measurement unit within the current reconstructed frame segment from the time-series light intensity signal. The four probe light intensity values ​​are composed of a first positive light intensity value, a first negative light intensity value, a second positive light intensity value, and a second negative light intensity value. Perform a difference operation between the first positive light intensity value and the first negative light intensity value to obtain a first-order moment difference detection value, and determine the first-order moment difference detection value as the first-order moment value corresponding to the current reconstructed frame segment; Perform a difference operation between the second positive light intensity value and the second negative light intensity value to obtain the second-order central moment difference detection value, and determine the second-order central moment difference detection value as the second-order central moment value corresponding to the current reconstructed frame segment; Extract the light intensity detection value corresponding to the fully bright pattern in the Hadamard modulation pattern within the current reconstructed frame segment, and determine the light intensity detection value corresponding to the fully bright pattern as the zero-order moment value corresponding to the current reconstructed frame segment.

6. The method as described in claim 1, characterized in that, Determining the centroid coordinates of the moving target in the current reconstructed frame based on the zeroth and first moments after background compensation specifically includes: Subtract the pre-calibrated static background zero-moment value from the zero-moment value corresponding to the current reconstructed frame segment to obtain the zero-moment value after background compensation. The first moment value corresponding to the current reconstructed frame segment is compared with the zero moment value after background compensation in a dimension-wise ratio operation to obtain the centroid abscissa and centroid ordinate of the moving target in the current reconstructed frame segment. The centroid coordinates of the moving target in the current reconstructed frame are formed by the centroid abscissa and the centroid ordinate.

7. The method as described in claim 1, characterized in that, The correlation calculation between the temporal light intensity signal and the motion-compensated reconstructed modulation pattern to reconstruct a dynamic scene image sequence with motion blur eliminated specifically includes: Extract the sequence of Hadamard probe light intensity values ​​corresponding to the Hadamard modulation patterns of each frame in the current reconstructed frame segment from the time-series light intensity signal; Perform an inner product operation between each probe light intensity value in the sequence of Hadamard probe light intensity values ​​and the modulation pattern of the corresponding frame in the motion-compensated reconstructed modulation pattern to obtain the one-dimensional Hadamard transform coefficient vector corresponding to the current reconstructed frame segment. Perform an inverse Hadamard transform on the one-dimensional Hadamard transform coefficient vector to reconstruct a single frame image corresponding to the reconstructed frame segment in the dynamic scene image sequence with motion blur eliminated.

8. A motion-compensated single-pixel real-time imaging system for dynamic scenes, used to execute the motion-compensated single-pixel real-time imaging method for dynamic scenes as described in any one of claims 1 to 7, characterized in that, The system includes: The acquisition module is used to project a set of geometric moment modulation patterns that have undergone differential normalization processing onto a dynamic scene containing moving targets, and to acquire the temporal light intensity signal reflected back from the dynamic scene. The geometric moment modulation patterns and the Hadamard modulation patterns used for image reconstruction are interleaved in a preset ratio to form a composite modulation sequence. The processing module is used to calculate the zero-order moment value, first-order moment value, and second-order central moment value corresponding to the current reconstruction frame segment from the time-series light intensity signal, determine the centroid coordinates of the moving target in the current reconstruction frame segment based on the zero-order moment value and first-order moment value after background compensation, and determine the rotation angle of the moving target in the current reconstruction frame segment based on the centroid coordinates and the second-order central moment value obtained by normalized difference processing. The processing module is further configured to perform a reverse shift operation on the Hadamard modulation pattern of the current reconstructed frame segment in the composite modulation sequence using the motion offset parameter formed by the centroid coordinates and the rotation angle, to generate a motion-compensated reconstructed modulation pattern. The execution module is used to perform correlation calculations between the temporal light intensity signal and the motion-compensated reconstructed modulation pattern to reconstruct a dynamic scene image sequence with motion blur eliminated.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to retrieve the code and execute the motion-compensated single-pixel real-time imaging method for dynamic scenes as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the motion-compensated single-pixel real-time imaging method for dynamic scenes as described in any one of claims 1 to 7.

Images