Adaptive factor micro-variation signal calculation amplification method in monogenic scale space

Abstract

The application discloses a method capable of extracting and calculating a target micro-variable signal in an amplified mixed signal, and comprises the following steps: expanding a two-dimensional sequence mixed signal into a plurality of sub-bands in a Poisson difference scale space; constructing a conjugate scale space of the two-dimensional sequence signal based on a Riesz transform of different spatial scale sub-bands, and jointly forming a mono-evolution scale space with the Poisson difference scale space; constructing a quaternion mono-evolution phase by using the direction and phase of the mono-evolution signal at each scale to replace the local phase, so as to eliminate the local phase sign uncertainty and the winding problem; processing two components of the quaternion phase of different scales through a time band-pass filter to obtain a phase difference of a micro-variable signal of interest, multiplying the two components of the filtered quaternion mono-evolution phase by an amplification factor of the current scale to calculate the phase of the target micro-variable signal extracted by amplification filtering; adaptively adjusting the amplification factor at the corresponding scale according to the wavelength of the scale filter during the calculation amplification process; and reconstructing the micro-variable signal of the calculation amplification by phase shift transformation of the micro-variable signal of the amplification. The application is suitable for micro-motion calculation amplification in optical measurement video, astronomical target luminosity micro-variable signal calculation amplification in astronomical photometry signal and the like, and has the characteristics of high calculation efficiency, low amplification distortion and wide application object.

Description

Technical Field

[0001] This invention relates to the fields of multidimensional signal processing and video processing technology, and in particular to a method for calculating and amplifying micro-variable signals encapsulated in multidimensional signals. Background Technology

[0002] Computational amplification of microsignals, a form of "computational microscopy," has made it possible to observe and study important biological or dynamic systems that are imperceptible to the naked eye. In recent years, scholars in various fields have begun to utilize video amplification technology as an auxiliary tool to aid scientific research and development; simultaneously, it has also opened up a wealth of applications in our daily lives. In the biomedical field, with the increasing global population and aging society, the demand for inexpensive, efficient, and automated healthcare solutions is growing. Computational amplification of microsignals can obtain vital signs information from video surveillance data in a non-contact manner, especially for premature newborns and the elderly. Moreover, continuous or intensive monitoring outside the clinical setting can not only provide doctors with timely samples but also provide long-term trends and statistical analyses as supplementary diagnostic evidence. In the field of engineering mechanics, functional infrastructure, including transportation, energy, and construction, is crucial to a country's economic production and the healthy development of cities. Due to natural degradation and potential damage, regular inspection and maintenance are essential to ensure the infrastructure operates at full capacity. Micro-signal computational amplification can separate vibration modes at different frequencies and monitor facility status through state monitoring videos acquired by optical cameras without affecting structural vibration. In the fields of intelligent monitoring and autonomous driving, micro-signal computational amplification technology is used to amplify micro-expressions and improve the recognition rate of fatigue and physiological abnormalities. In the field of space target surveillance, photometric micro-change computational amplification is an important technical support for sensing the characteristic parameters of space targets.

[0003] This invention discloses an adaptive factor-based amplification method for small-signal computation in a single-scale space. It converts a two-dimensional sequence signal into a supercomplex scale space within the single-scale space. By analyzing the local amplitude and local quaternary phase of the sub-band signal at different scales in the complex scale space, a smoothness filter is constructed using the third-order changes of the local amplitude and local quaternary phase to evaluate the smoothness of the time series data. Small-signal computation is distinguished from the second-order changes in the local amplitude and local quaternary phase. An adaptive amplification factor is set at different scales. Micro-amplification is achieved by amplifying the second-order local phase difference, and photometric micro-amplification is achieved by amplifying the second-order local amplitude difference. Summary of the Invention

[0004] The purpose of this invention is to adaptively set the amplification factor to amplify minute changes in multidimensional signals to a level that is easily perceptible to the naked eye, while facilitating subsequent calculation, processing, and analysis of signal characteristics. Furthermore, this invention can complete the amplification task under various imaging conditions.

[0005] The technical solution adopted to achieve the purpose of this invention is:

[0006] S1: Based on the two-dimensional Poisson kernel, decompose each frame of data in the two-dimensional signal sequence into multiple sub-band signals with different spatial frequencies, and construct a Poisson difference scale spatial representation.

[0007] S2: The signals of each sub-band in the Poisson difference scale space and their two components of the Riesz transform together form a single-evolution signal, thus forming a single-evolution scale space;

[0008] S3: Extract the local amplitude and local quaternary phase of each sub-band signal in the single-stage scale space;

[0009] S4: Based on the time-domain filter, extract the second and third order changes of the quaternary phase signal, and perform local amplitude-weighted Gaussian denoising to enhance the signal-to-noise ratio of the subband signal;

[0010] S5: Construct a smoothness filter based on the third-order change quantity to extract the second-order micro-change information of the quaternary phase signal;

[0011] S6: Adaptively set the amplification factor at different scales, calculate the amplified and filtered quaternary phase second-order micro-variable signal, reconstruct the two-dimensional signal sequence through phase shift transformation, and obtain the two-dimensional signal sequence with amplified micro-variable calculation.

[0012] Compared with the prior art, the advantages and innovations of this invention are:

[0013] This invention processes multidimensional signals within a single-scale spatial framework, ensuring that all sub-bands have the same resolution, thus avoiding information loss during upsampling and downsampling operations in image pyramids.

[0014] This invention adaptively sets the sub-band signal amplification factor within a single-scale spatial framework to avoid spurious signals arising from computational amplification.

[0015] This invention reduces the additional noise caused by time-domain filtering by employing local amplitude-weighted smoothing processing, thereby improving the fidelity of the signal after computational amplification.

[0016] This invention is applicable to video surveillance and video measurement scenarios, as well as to other multidimensional signals such as the calculation and amplification of target photometric micro-variation signals in space target monitoring. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for calculating and amplifying adaptive factor micro-variable signals in a single-scale space according to the present invention.

[0018] Figure 2 This is a flowchart illustrating the steps of an adaptive factor micro-variation signal calculation and amplification method in a single-scale space according to the present invention. Detailed Implementation

[0019] The specific implementation steps of the present invention are described in detail below with reference to the accompanying drawings. All technical and scientific terms used have the same meaning as commonly understood by one of ordinary skill in the art described in this invention.

[0020] S1: Based on the two-dimensional Poisson kernel, decompose each frame of data in the two-dimensional signal sequence into multiple sub-band signals with different spatial frequencies, and construct a Poisson difference scale spatial representation.

[0021] S2: The signals of each sub-band in the Poisson difference scale space and their two components of the Riesz transform together form a single-evolution signal, thus forming a single-evolution scale space;

[0022] S3: Extract the local amplitude and local quaternary phase of each sub-band signal in the single-stage scale space;

[0023] S4: Based on the time-domain filter, extract the second and third order changes of the quaternary phase signal, and perform local amplitude-weighted Gaussian denoising to enhance the signal-to-noise ratio of the subband signal;

[0024] S5: Construct a smoothness filter based on the third-order change quantity to extract the second-order micro-change information of the quaternary phase signal;

[0025] S6: Adaptively set the amplification factor at different scales, calculate the amplified and filtered quaternary phase second-order micro-variable signal, reconstruct the two-dimensional signal sequence through phase shift transformation, and obtain the two-dimensional signal sequence with amplified micro-variable calculation.

[0026] In the above scheme, step S1 is to construct the Poisson difference scale space representation of the two-dimensional signal, which specifically includes the following process:

[0027] S11: If the input two-dimensional signal sequence is a color image sequence, convert each frame of the image sequence to the YIQ color space and decompose the luminance channel signal. In this invention, a two-dimensional Poisson kernel is used to perform a convolution operation on the luminance image. This two-dimensional Poisson kernel can be considered an extension of the one-dimensional Cauchy probability density function.

[0028]

[0029] Where x = (x, y) represents the spatial coordinates, and s is the scale parameter. The frequency domain representation of the Poisson kernel is:

[0030] P(ξ;s)=exp(-2π|ξ|s)

[0031] Where ξ = [ω1, ω2] represents the frequency domain coordinates. Forlsberg proved that the Poisson kernel satisfies the definition criteria for linear scale space proposed by Iijima (see reference: M. Forlsberg. The Monogenic Scale-Space: A Unifying Approach to Phase-Based Image Processing in Scale-Space[J]. Journal of Mathematical Imaging & Vision, 2004, 21(1-2): 5-26.), that is, the Poisson kernel can also establish a linear scale space.

[0032] S12: Construct the Poisson scale space. For color image sequences, construct the Poisson scale space based on the luminance channel signal. The Poisson filter kernel for continuously varying scale parameters of a two-dimensional signal sequence is:

[0033]

[0034] Where s m Let be the m-th scale parameter, and s m >s m-1 I F (x,y,t) represents the Fourier transform of the two-dimensional signal I(x,y,t), where t represents the signal data of the t-th frame. Inverse Fourier Transform

[0035] S13: The Poisson difference space is constructed by subtracting the two-dimensional subband signals of two adjacent Poisson-filtered frames, which contains multiple subband signals with different spatial frequencies.

[0036]

[0037] And a low-pass co-quantum band:

[0038]

[0039] Where DoP(ξ;d) i ) represents the i-th Poisson difference filter kernel, I d (x,y,t,d i ) represents the Poisson difference space subband of the i-th layer.

[0040] In the above scheme, step S2 involves constructing a single-scale representation of the two-dimensional signal based on the Poisson difference scale space representation of the two-dimensional signal. Specifically, this includes the following process:

[0041] S21: Riesz transform Poisson difference scale space sub-band signals;

[0042] The Riesz transform is a multidimensional extension of the Hilbert transform, possessing directional controllability. In two dimensions, the frequency domain representation of the Riesz transform is:

[0043]

[0044] in(·,·) T I represents the transpose of a vector. F Let I be the Fourier transform of the original two-dimensional signal I, where ω = [ω1, ω2] represents the frequency domain coordinates. Then, the spatial expression of the Riesz transform result is:

[0045]

[0046] in This is the inverse Fourier transform.

[0047] The original two-dimensional signal is taken as the real part, and its two components of the Riesz transform are taken as the imaginary part. A single-evolution signal is constructed using this hypercomplex number:

[0048] I M (x)=I(x)-iR1(x)-jR2(x)

[0049] Where x = [x, y] represents spatial coordinates. The original signal and the two components of the Riesz transform constitute a three-dimensional signal vector R = (I, R1, R2).

[0050] S22: The sub-band signals of each scale in the Poisson difference scale space and their Riesz transform components constitute the single-evolution signals at each scale. All the single-evolution signals of the sub-bands form the single-evolution scale space representation of the two-dimensional signal.

[0051] In the above scheme, step S3 extracts the local amplitude and local quaternary phase of each sub-band signal in the single-evolution scale space. The specific process is as follows:

[0052] Two-dimensional signal I in the l-th sub-band of single-scale space d (x,y,t,d l The Riesz transform of ) is vector It can be determined by the local amplitude A l Local phase and local direction θ l Represented as:

[0053]

[0054] Using quaternion phase to simultaneously represent local phase and local direction avoids the problems of phase sign uncertainty and entanglement:

[0055]

[0056] Where t represents the number of frames in the two-dimensional signal sequence, l represents the number of sub-band layers, and θ t l Let represent the local principal direction of the l-th sub-band of the t-th frame of the two-dimensional signal at point (x,y). This represents the phase along the local principal direction. The local phases used in this invention are all solved along the local principal direction; for ease of representation, quaternion phases are used. Abbreviated as below

[0057] In the above scheme, step S4 involves performing time-domain filtering on the quaternary phase, extracting the second and third order changes of the quaternary phase respectively, and performing local amplitude-weighted Gaussian denoising. The specific process is as follows:

[0058] S41: Extract the second and third order changes of the quaternary phase.

[0059] Second-order information of the quaternary phase of each subband in the scale space of multiple adjacent two-dimensional signals is extracted within a time-domain sliding window with a width of 2w. and third-order information

[0060]

[0061] in The quadrature phase at (x,y) in the l-th subband of the two-dimensional signal across multiple frames (from frame t-w+1 to frame t+w) within a sliding window. For convolution operations, G σ (t) is a Gaussian filter with variance σ. 2 .

[0062] S42: Weighted smoothing processing to improve signal-to-noise ratio

[0063] To reduce the noise introduced by time-domain filtering, the second and third order changes of the quaternary phase are subjected to amplitude-weighted smoothing to improve the signal-to-noise ratio.

[0064]

[0065]

[0066] in These are the two components after quaternary phase weighting and smoothing. These are the two components of the smoothness filter.

[0067] In the above scheme, step S5 involves constructing a smoothness filter based on the third-order change quantity to extract the second-order micro-change information of the quaternary phase signal. The specific process is as follows:

[0068] A smoothness filter constructed using third-order change information filters the second-order change information of phase signals in different sub-bands, filtering out rapidly changing signal regions and distinguishing between signals with large and small changes. Taking one component of the two imaginary parts of the quaternary phase as an example, the formula is as follows:

[0069]

[0070]

[0071] in, and Let $\frac{x}{y}{t}$ represent the minimum and maximum values ​​of the third-order phase transformation modulus of the four-element phase in the subband signal of layer $l$ in frame $t$. The constructed smoothness filter is $smoothness1(x,y,t,d)$. l It has the same size as the sub-band signal.

[0072] In the above scheme, step S6 involves adaptively setting amplification factors at different scales to obtain a two-dimensional signal sequence with amplified micro-changes. The specific process is as follows:

[0073] S61: In the smoothness filter, set a variable exponent (β > 0) parameter to adjust the weight of signals with large amplitude changes and signals with small amplitude changes.

[0074] JAF1(x,y,t,d l = smoothness1(x,y,t,d) l ) β

[0075] The filter is corrected layer by layer, transferring the information from the coarse subband of the signal with large amplitude variation to the fine subband layer by layer:

[0076]

[0077] Where l is the number of subband layers to be corrected, N is the total number of layers used for layer-by-layer correction, and Π· represents the cumulative multiplication operation.

[0078] S62: The second-order transformation information of the quaternary phase consists of two components, which are smoothed by the smoothness filter pJAF1(x,y,t,d) of each sub-band. l ), pJAF2(x,y,t,d l Filtering is performed separately to obtain the quaternion phase information to be amplified:

[0079]

[0080]

[0081] S63: Adaptively sets the magnification factor at different scales. Two adjacent scale parameters s m >s m-1 The wavelength of the Poisson differential filter is:

[0082]

[0083] Let the width of the impulse response of the Poisson differential filter be τ = ηλ. m η is the scaling factor. The amplification factor of the corresponding sub-band small-signal is adjusted according to the wavelength of the sub-band filter at each single-evolution scale.

[0084]

[0085] Here, α is the user-preset magnification factor, and λ c It is the cutoff wavelength parameter.

[0086] S64: Calculate the second-order phase change information of the four elements after amplification and filtering, perform phase shift transformation, reconstruct the two-dimensional signal sequence, and obtain the amplified two-dimensional signal sequence with micro-change calculation.

[0087] Micro-variation signals of each sub-band in single-scale space and Amplify α m Times, the same as the current frame sub-band vector Perform phase shift transformation:

[0088]

[0089] Taking the real part of the above equation, we obtain the motion-amplified sub-band signal:

[0090]

[0091] This process is applied to each sub-band signal in the single-scale space. Then, the low-pass margin and all bandpass sub-bands are accumulated to reconstruct a two-dimensional signal sequence with computational amplification of the micro-signal. If it is a color image sequence, the two-dimensional signal sequence with computational amplification of the micro-signal is combined with the chromaticity information of the YIQ color space in step S11 to reconstruct the computationally amplified image sequence.

[0092] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. All other embodiments obtained by those skilled in the art based on the technical solutions of the present invention without creative effort are within the scope of protection of the present invention.

Claims

1. A method for calculating and amplifying adaptive factor small-variation signals in a single-scale space, characterized in that, The process involves converting a two-dimensional sequence signal to a hypercomplex scale space within a single-scale space. By analyzing the local amplitude and local quaternary phase of the sub-band signal at different scales in the hypercomplex scale space, a smoothness filter is constructed using the third-order changes of the local amplitude and local quaternary phase to evaluate the smoothness of the time series data. This process distinguishes subtle signals within the second-order changes of local amplitude and local quaternary phase, adaptively setting amplification factors at different scales, achieving micro-amplification by amplifying the second-order local phase difference, and achieving micro-amplification of photometric intensity by amplifying the second-order local amplitude difference. The steps include: S1: Based on the two-dimensional Poisson kernel, decompose each frame of data in the two-dimensional signal sequence into multiple sub-band signals with different spatial frequencies, and construct a Poisson difference scale spatial representation. S2: The signals of each sub-band in the Poisson difference scale space and their two components of the Riesz transform form a single-evolution signal, thus forming a single-evolution scale space; S3: Extract the local amplitude and local quaternary phase of each sub-band signal in the single-scale space; S4: Based on the time-domain filter, extract the second and third order changes of the quaternary phase signal and perform local amplitude-weighted Gaussian denoising to enhance the signal-to-noise ratio of the subband signal; S5: Construct a smoothness filter based on the third-order change quantity to extract the second-order micro-change information of the quaternary phase signal; S6: Adaptively set the amplification factor at different scales, calculate the amplified and filtered quaternary phase second-order micro-variable signal, reconstruct the two-dimensional signal sequence through phase shift transformation, and obtain the two-dimensional signal sequence amplified by micro-variable calculation; Based on the wavelength of the Poisson difference filter, the adaptive amplification factor is calculated, and the two adjacent scale parameters s are used. m >s m-1 The wavelength of the Poisson differential filter is: Let the width of the impulse response of the Poisson differential filter be τ = ηλ m η is a scaling factor, which adjusts the amplification factor of the corresponding sub-band small-signal based on the wavelength of each single-scale sub-band filter. Here, α is the user-preset magnification factor, and λ c It is the cutoff wavelength parameter; The specific steps of S1 are as follows: The process includes the following steps: S11: If the input two-dimensional signal sequence is a color image sequence, convert each frame of the image sequence to the YIQ color space, decompose the luminance channel signal, and set the two-dimensional Poisson kernel parameters. The Poisson kernel spatial domain representation is as follows: Where x = [x, y] represents the spatial coordinates of a frame of data in a two-dimensional signal sequence, s is the scale parameter, and the frequency domain representation of the Poisson kernel is: P(ξ;s)=exp(-2π|ξ|s) Where ξ = [ω1, ω2] represents the frequency domain coordinates; S12: Construct a Poisson scale space. If it is a color image sequence, construct the Poisson scale space based on the brightness channel signal. The Poisson filter kernel for continuously varying scale parameters of a two-dimensional signal sequence is: Among them, s m Let I be the m-th scale parameter. F (x,y,t) represents the Fourier transform of the two-dimensional signal I(x,y,t), where t represents the signal data of the t-th frame. This is the inverse Fourier transform; S13: The Poisson difference space is constructed by subtracting the two-dimensional subband signals of two adjacent Poisson-filtered frames, which contains multiple subband signals with different spatial frequencies. And a low-pass co-quantum band: All subbands have the same resolution, DoP(ξ;d i Let I be the i-th Poisson difference filter kernel. d (x,y,t,d i ) represents the Poisson difference space subband of the i-th layer.

2. The method for calculating and amplifying adaptive factor small-variation signals in a single-scale space as described in claim 1, characterized in that, Step S2 specifically includes the following process: S21: Riesz transform Poisson difference scale space sub-band signals; The frequency domain representation of the Riesz transform is: in(·,·) T I represents the transpose of a vector. F The Fourier transform of the original two-dimensional signal I is given, where ω = [ω1, ω2] represents the frequency domain coordinates. The original two-dimensional signal is taken as the real part, and its two components of the Riesz transform are taken as the imaginary part. This hypercomplex number is used to construct a single-evolution signal: I M (x)=I(x)-iR1(x)-jR2(x) Where x = [x, y] represents spatial coordinates, and the original two-dimensional signal and the two components of the Riesz transform constitute a three-dimensional signal vector R = (I, R1, R2); S22: Riesz transform is performed on the sub-bands of each scale in the Poisson difference scale space to obtain the single-evolution signals at each scale, which together form the single-evolution scale space.

3. The method for calculating and amplifying adaptive factor small-variation signals in a single-scale space as described in claim 1, characterized in that, The specific process of step S3 is as follows: Two-dimensional signal I in the l-th sub-band of single-scale space d (x,y,t,d l The Riesz transform of ) is ,vector It can be determined by the local amplitude A l Local phase and local direction θ l Represented as: Quaternion phase is used to represent local phase and local direction simultaneously, avoiding phase sign uncertainty and entanglement problems: Where t represents the number of frames in the two-dimensional signal sequence, l represents the number of sub-band layers, and θ t l Let represent the local principal direction of the l-th sub-band of the t-th frame of the two-dimensional signal at point (x,y). The quaternion phase represents the phase along the local principal direction. All local phases used are solved along the local principal direction. For ease of representation, the quaternion phase is used. Abbreviated as below 4. The method for calculating and amplifying adaptive factor small-variation signals in a single-scale space as described in claim 1, characterized in that, The specific process of step S4 is as follows: The quaternary phase is filtered in the time domain to extract the second and third order changes of the quaternary phase, and then local amplitude-weighted Gaussian denoising is performed. Second-order information of the quaternary phase of each subband in the scale space of multiple adjacent two-dimensional signals is extracted within a time-domain sliding window with a width of 2w. and third-order information in, The quadrature phase at (x,y) in the l-th subband of a multi-frame two-dimensional signal within a sliding window. For convolution operations, G σ (t) is a Gaussian filter with variance σ. 2 ; To reduce the noise introduced by time-domain filtering, the second and third order changes of the quaternary phase are subjected to amplitude-weighted smoothing to improve the signal-to-noise ratio. in, These are the two components after quaternary phase weighting and smoothing. These are the two components of the smoothness filter.

5. The method for calculating and amplifying adaptive factor small-variation signals in a single-scale space as described in claim 1, characterized in that, The specific process of step S5 is as follows: A smoothness filter constructed using a third-order transformation filters the second-order transformation information of phase signals in different sub-bands, filtering out rapidly changing signal regions and distinguishing between signals with large and small changes. Taking one component of the two imaginary parts of the quaternary phase as an example, the formula is as follows: in, and The smoothness filter smoothness1(x,y,t,d) represents the minimum and maximum values ​​of the third-order phase transformation modulus of the four-element phase in the l-th layer subband signal of frame t. l It has the same size as the sub-band signal.

6. The method for calculating and amplifying adaptive factor small-variation signals in a single-scale space as described in claim 1, characterized in that, The specific process of step S6 is as follows: S61: In the smoothness filter, set a variable exponential parameter β to adjust the proportion of signals with large changes to those with small changes. JAF1(x,y,t,dl)=smoothness1(x,y,t,d l ) β The filter is corrected layer by layer, transferring the information from the coarse subband of the signal with large variations to the fine subband layer by layer: Where l is the number of subband layers to be corrected, N is the total number of layers used for layer-by-layer correction, and ∏ represents the cumulative multiplication operation; S62: The second-order transformation information of the quaternary phase consists of two components, which are smoothed by the smoothness filter pJAF1(x,y,t,d) of each sub-band. l ), pJAF2(x,y,t,d l Filtering is performed separately to obtain the quaternion phase information to be amplified: S63: Adaptive setting of magnification factor at different scales; S64: Calculate the second-order phase change information of the four-element phase after amplification and filtering, perform phase shift transformation, reconstruct the two-dimensional signal sequence, and obtain the micro-change amplified two-dimensional signal sequence; Micro-variation signals of each sub-band in single-scale space and Amplify α m The phase shift transformation is performed on the current frame subband vector, multiplied by 1. Taking the real part of the above equation, we obtain the sub-band signal after micro-amplification: This process is applied to each sub-band signal in the single-scale space. Then, the low-pass margin and all bandpass sub-band signals are accumulated to reconstruct a two-dimensional signal sequence with amplified micro-signal. If it is a color image sequence, the two-dimensional signal sequence with amplified micro-signal is combined with the chromaticity information of the YIQ color space in step S11 to reconstruct the amplified image sequence.

Images