Spray image recognition and measurement method based on three-frame alignment motion compensation FFDNet

Abstract

This invention discloses an FFDNet spray image recognition and measurement method based on three-frame alignment motion compensation, comprising: acquiring three adjacent frames of a spray image; estimating a first displacement field and a second displacement field based on a lightweight optical flow pyramid algorithm to generate two aligned frames corresponding to the target frame for motion compensation; inputting the target frame, the two aligned frames, and a spatial noise map into an improved FFDNet denoising network to output a denoised image and a high-frequency feature map; fusing the denoised image and the high-frequency feature map to generate a spray edge probability map and performing thin edge refinement to generate a spray thin boundary map; extracting and fitting the boundary point set of the spray thin boundary map; and measuring the spray geometric parameters. The structural design of this measurement method is conducive to balancing noise suppression and edge preservation of the spray image with low computational cost, providing high-quality input for stable measurement of spray geometry, reducing ghosting and jitter of the spray motion, and improving the accuracy of spray image recognition.

Description

Technical Field

[0001] This invention relates to the field of image technology, specifically to an FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation. Background Technology

[0002] Spray images are often subject to interference from multiple sources, including stripe and speckle noise, and light sources. Simultaneously, spray boundaries exhibit thin, filamentous, and flickering temporal characteristics. Traditional denoising methods typically involve "single-frame denoising / segmentation, post-processing fitting, and geometric measurement," which suffers from several influencing factors: 1. Temporal instability: Independent frame-by-frame processing leads to significant jitter in the cone angle / penetration curve; 2. Thin filaments are prone to blurring: General denoising tends to smooth high frequencies, damaging boundaries; 3. Task fragmentation: Upstream denoising aims for optimal PSNR / SSIM, rather than "geometric measurement optimization"; 4. Alignment deficiencies: Misaligned stacking of multi-frame information prevents effective utilization of temporal redundancy; 5. High annotation costs: Fine-grained ground truth is scarce, and end-to-end supervision is limited.

[0003] Chinese invention patent application CN112991235B discloses a video denoising method and a video denoising terminal. This patent proposes a two-level video denoising framework based on motion and stillness analysis. The process involves first inputting the "current frame + historical state frame" into a motion estimation network to obtain the motion / stillness probability of each region; then, the current frame is sent to the "pre-denoising module" of the denoising network for coarse denoising; subsequently, motion and stillness fusion is performed based on the motion and stillness analysis results; finally, a "post-denoising module" performs secondary fine denoising to suppress fine-grained noise and output the result. Another Chinese invention patent application CN113315884A discloses a real-time video denoising method, device, terminal, and storage medium. This patent first processes the "registration frame (denoising result of the previous frame)" and the "reference frame..." Image registration is performed on the current frame, followed by ghost detection (ghost mask generated by differential after smoothing). Temporal fusion denoising is then performed using this mask as weights. Subsequently, edge-preserving spatial denoising is applied to regions such as motion, detail, flatness, and brightness. The processed result is recursively used as the registration frame for the next frame, achieving stable denoising across multiple frames while maintaining video quality in detail, edges, and low-light scenes. The above denoising methods are only applicable to denoising processing of video frame series. However, for spray image processing, this denoising method cannot effectively utilize the temporal redundancy of the spray image sequence.

[0004] Therefore, there is an urgent need for a lightweight, real-time, time-stable, and geometrically optimal end-to-end method. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A spray image recognition and measurement method based on three-frame aligned motion compensation using FFDNet includes the following steps and contents:

[0008] S1. Acquire three adjacent frames of the spray image, including the target frame L. t Previous frame L t-1 and the following frame L t+1 ;

[0009] S2. Estimate the L of the previous frame based on the lightweight optical flow pyramid algorithm. t-1 To target frame L t The first displacement field and the subsequent frame L t+1 To target frame L t The second displacement field, and based on the first and second displacement fields, respectively, the previous frame L t-1 and the following frame L t+1 Motion compensation is performed to generate a frame L that matches the target frame. t The corresponding first alignment frame L1 and second alignment frame L2;

[0010] S3, based on target frame L t Generate a spatially variable noise map and set the target frame L t The first aligned frame L1, the second aligned frame L2, and the spatially variable noise map are used as multi-channel input features to improve the FFDNet denoising network, outputting a denoised image, and the high-frequency feature map is output by the shallow feature bypass of the improved FFDNet denoising network.

[0011] S4. The denoising features of the denoised image and the high-frequency features of the high-frequency feature image are fused to generate a spray edge probability map. The spray edge probability map is then refined into a thin edge map by a learnable nonmaximum suppression module.

[0012] S5. Based on nozzle structural parameter constraints, scan and extract the left and right spray boundary point sets of the spray thin boundary map along the spray diffusion direction, fit the boundary straight line, and measure the spray geometric parameters.

[0013] According to one aspect of this disclosure, the spatially variable noise map is a noise intensity distribution map generated based on a differential weighting strategy, wherein lower noise weights are assigned to pixel groups with high gradients at the edges, and higher noise weights are assigned to pixel groups in high variance or overexposed areas.

[0014] According to one aspect of this disclosure, the method for generating the first alignment frame L1 and the second alignment frame L2 is as follows:

[0015] S21, For target frame L t Previous frame Lt-1 and the following frame L t+1 Downsample to the first scale, calculate the coarse displacement field at the first scale, and obtain the initial optical flow;

[0016] S22. Upsample the initial optical flow to the second scale and construct a local correlation window near the aligned pixel pairs. Calculate the residual displacement based on the local correlation window.

[0017] S23. Superimpose the initial optical flow and residual displacement to obtain the first displacement field and the second displacement field;

[0018] S24. Based on the first displacement field and the second displacement field, the previous frame L is resampled using the reverse resampling method. t-1 and the following frame L t+1 Mapped to target frame L respectively t Using the coordinate system, obtain the first alignment frame L1 and the second alignment frame L2.

[0019] According to one aspect of this disclosure, the target frame L t The first aligned frame L1, the second aligned frame L2, and the spatially variable noise map are concatenated along the channel dimension to form a four-channel input feature, which is then input into the improved FFDNet denoising network. The improved FFDNet denoising network includes downsampling layers, convolutional feature extraction layers, spatial modulation layers, and upsampling reconstruction layers; its denoising methods include:

[0020] S31, the downsampling layer performs feature downsampling on the four-channel input features;

[0021] S32, the convolutional feature extraction layer extracts spray structure features, noise features, and temporal redundancy features;

[0022] S33. The spatial modulation layer concatenates the noise feature map and the temporal feature map, and generates spatial modulation coefficients to weight and adjust the feature responses of different spatial regions. The noise feature map is generated based on convolutional encoding of a spatially variable noise map, and the temporal feature map is generated by calculating the first aligned frame L1 and the target frame L1 respectively. t and the second aligned frame L2 and the target frame L t Output of difference information;

[0023] S34. Upsample and reconstruct the denoised feature map after denoising, and output a denoised image that incorporates temporal redundancy information.

[0024] According to one aspect of this disclosure, the shallow feature side route is derived from the shallow convolutional block of the convolutional feature extraction layer, and after convolutional dimensionality reduction and upsampling processing, a high-frequency feature map with the same size as the denoised image is output.

[0025] According to one aspect of this disclosure, the method for generating a spray edge probability map is as follows: the denoised image and the high-frequency feature map are respectively feature-encoded, and a fused feature map is generated through a gated fusion mechanism. The gated fusion mechanism is used to increase the fusion weight of the high-frequency feature map in the spray edge region and increase the fusion weight of the denoised image in the background region; the spray edge probability map is output based on the fused feature map.

[0026] According to one aspect of this disclosure, the learnable nonmaximum suppression module includes a gradient direction estimation unit, a neighborhood response comparison unit, and a learnable threshold unit; the gradient direction estimation unit is used to calculate the gradient direction of the spray edge probability map; the neighborhood response comparison unit is used to compare the response values ​​of neighboring pixels along the gradient direction; and the learnable threshold unit is used to filter candidate boundary pixels according to the learnable threshold to output a spray thin boundary map.

[0027] According to one aspect of this disclosure, the boundary line fitting process in step S5 employs a differentiable singular value decomposition fitting method.

[0028] According to one aspect of this disclosure, it also includes step S6: constructing a multi-task loss function with the primary objective of optimizing spray geometry measurement and performing backpropagation optimization.

[0029] Multi-task loss function: ;

[0030] Where L is the total loss; , , and These are the weights of each loss, and L g For geometric parameter loss; L e For edge detection loss; L d Image denoising loss; L t The timing consistency loss is used to constrain the consecutive target frames L. t The smoothness of the curves showing the changes in spray cone angle and spray penetration distance.

[0031] Compared with existing technologies, the FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation of the present invention has the following beneficial effects:

[0032] 1. A two-level pyramid three-frame lightweight optical flow alignment is adopted to press adjacent frames onto the target frame coordinate system. The target frame, the aligned frame, and the spatially variable noise map are input into the FFDNet network to achieve multi-frame collaborative denoising and filament edge preservation under alignment domain constraints. This structural design balances denoising and edge preservation of spray images with low computational cost, providing high-quality input for stable measurement of subsequent spray geometry, reducing ghosting and jitter caused by spray motion, and improving the recognition accuracy of spray images.

[0033] 2. By using a differentiable closed-loop mechanism to backpropagate measurement errors and optimize the FFDNet network, unlike traditional video denoising that only pursues image quality, this method takes the optimal geometric measurement as the training and optimization objective, which significantly reduces the systematic error and timing jitter of geometric measurement. It is suitable for online monitoring of spray under various imaging conditions such as backlight and schlieren and for experimental bench applications. Attached Figure Description

[0034] Figure 1 This is the main flowchart of the FFDNet spray image recognition and measurement method disclosed in this invention;

[0035] Figure 2 This is a flowchart illustrating the operation of the FFDNet spray image recognition and measurement method disclosed in this invention.

[0036] Figure 3 This is a flowchart of the lightweight optical flow pyramid algorithm disclosed in this invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely the best embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] The term "embodiment" as used herein means that a particular method, step, or content described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0039] This embodiment provides a spray image recognition and measurement method based on three-frame aligned motion compensation using FFDNet. The aim is to train an FFDNet network to generate an adaptive spray image processing model that evaluates noise levels, removes noise from spray images, and accurately identifies spray images and spray information. Figures 1-3 As shown, it includes the following steps and contents:

[0040] S1. Acquire three adjacent spray images under the same spraying condition, including the target frame L. t Previous frame L t-1 and the following frame L t+1 Spray images can be obtained by combining a high-speed camera with backlight imaging, shadow imaging, or schlieren imaging.

[0041] Target frame L t To provide a reference frame for the final output denoised image and spray geometry parameters, the previous frame L... t-1 and the following frame L t+1 Used to provide target frame L t Temporal redundancy information between consecutive moments; since the real spray structure in the spray image has a certain continuity between adjacent frames, while random noise, speckle noise, stripe perturbation and local imaging interference usually do not have stable cross-frame consistency, using three adjacent frames as input can form bidirectional temporal constraints in the subsequent alignment domain, enhance the real spray boundary information and suppress random interference in the alignment domain, and improve the stability of multi-frame collaborative denoising and spray geometry measurement.

[0042] S2. Lightweight Optical Flow Pyramid Motion Compensation: To avoid ghosting at the spray boundary caused by directly fusing three frames, this embodiment uses a lightweight optical flow pyramid algorithm to perform motion compensation on the previous frame L. t-1 and the following frame L t+1 Perform motion compensation to align it to the target frame L. t coordinate system;

[0043] The lightweight optical flow pyramid algorithm achieves low-complexity optical flow estimation through few layers and multiple scales, enabling stable and accurate pixel-level cross-frame alignment with relatively low computational overhead. Specifically, as shown... Figure 3 As shown, the lightweight optical flow pyramid algorithm in this embodiment includes two stages: 1 / 8-scale coarse optical flow estimation and 1 / 4-scale residual correction. Motion compensation is performed through reverse resampling to generate aligned frames. The method is as follows:

[0044] S21, Move the target frame L t Previous frame L t-1 and the following frame L t+1 Downsampled to a 1 / 8 scale, the coarse displacement field is obtained through local feature matching or small window correlation calculation, yielding the previous frame L. t-1 To target frame L t The initial optical flow and the subsequent frame L t+1 To target frame L t The initial optical flow;

[0045] S22. Upsample the initial optical flow to a 1 / 4 scale, set a local correlation window centered on the pixel pairs aligned with the initial optical flow. The local correlation window can be a 3×3 or 5×5 window. Obtain the residual displacement through local correlation calculation, and superimpose the initial optical flow and the residual displacement to obtain the refined displacement field.

[0046] S23. Perform motion compensation based on the refined displacement field through reverse resampling, and generate an aligned frame; read the previous frame L. t-1 To target frame L tLet the first displacement field be the compensation amount of the first displacement field for pixel (x,y) as... Then the first aligned frame Read the following frame L t+1 To target frame L t The second displacement field, let the compensation amount of the second displacement field for pixel (x,y) be... Then the second alignment frame ;

[0047] Through the above processing, the spray structure of the previous and next frames is mapped to the target frame coordinate system, thereby reducing the multi-frame fusion ghosting caused by the spray front advancement and boundary expansion, and outputting the above two aligned frame images and their alignment domain.

[0048] S3, Move the target frame L t The first aligned frame L1, the second aligned frame L2, and the spatially variable noise map n-map are used as four-channel inputs to improve the FFDNet denoising network for fusion and denoising.

[0049] In this embodiment, the spatially variable noise map (n-map) is a noise map generation model based on the target frame L. t Generates indicators to show the denoising intensity of different pixel regions;

[0050] In one possible embodiment, the noise map generation model learns information such as the exposure saturation region and the intensity of row and column stripes in the image to remove structural interference. Finally, based on the statistical information of local variance, gradient magnitude, and exposure intensity, an n-map is constructed for each pixel. The noise map generation model constructs the n-map based on a differential weighting strategy. Specifically, the edges of fine filaments contribute the most to geometric measurements and need to suppress excessive denoising to preserve fidelity. Far-field droplets, speckle, or overflow areas require stronger denoising to reduce false edges. Therefore, the noise map generation model assigns lower noise weights to high gradient regions at the edges and higher noise weights to local high variance regions, speckle regions, stripe interference regions, or overexposed regions. This allows the FFDNet denoising network to adaptively adjust the spatial denoising intensity according to the input of the n-map, thereby balancing the spray image processing goals of global denoising and local edge preservation.

[0051] In this embodiment, the target frame L t The first aligned frame L1, the second aligned frame L2, and the n-map are concatenated along the channel dimension to form a four-channel input feature. The improved FFDNet denoising network is input into the improved FFDNet denoising network, which includes downsampling layers, convolutional feature extraction layers, spatial modulation layers, upsampling reconstruction layers, and shallow feature bypasses.

[0052] S31. The downsampling layer performs feature downsampling on the four-channel feature map, reducing the feature map size and improving computational efficiency.

[0053] S32, the convolutional feature extraction layer extracts spray structure features, noise features and temporal redundancy features. The convolution operation uses a 3×3 convolution kernel and completes nonlinear feature extraction through the ReLU activation function.

[0054] In some possible embodiments, a frequency domain cueing layer is introduced at the first layer of the convolutional network, which can be implemented using a DCT frequency domain cueing layer. The four-channel feature map is transformed by DCT to obtain a frequency domain feature map. The main frequency band range of schlieren noise is statistically determined, and the coefficients of the corresponding main frequency band in the frequency domain feature map are suppressed. At the same time, the suppression strength is adaptively adjusted through a gating mechanism. The gating parameters are generated by the frequency domain feature map after global average pooling and activation operations. Finally, the processed frequency domain feature map is subjected to inverse DCT transformation to obtain a frequency domain enhanced feature map, which is then input into the subsequent lightweight convolutional layer to suppress the main frequency noise of the schlieren image and reduce the computational overhead of the subsequent network.

[0055] The convolutional feature extraction layer consists of multiple lightweight convolutional layers stacked together to extract subtle features, such as spray filaments and particle details. Each convolutional block in this stage enhances the information pathway by introducing residual connections, avoiding gradient vanishing during network training, while preserving the structural features of the aligned frames.

[0056] S33. The spatial modulation layer concatenates the noise feature map with the temporal feature map, and generates spatial modulation coefficients through 1×1 convolution and a sigmoid activation function. These coefficients are used to adaptively adjust the denoising intensity for different image regions, specifically for the target frame L. t Denoising is performed; specifically, n-map convolutional encoding is used to generate noise feature maps, and the first aligned frame L1 and the target frame L1 are calculated respectively. t and the second aligned frame L2 and the target frame L t The difference information is used to output a temporal feature map; the noise intensity feature map and the temporal feature map are dimensionally concatenated to obtain a guided feature map, and a dual-guided denoising mechanism is constructed. The spatial modulation coefficients for denoising in different regions are generated by 1×1 convolution and Sigmoid activation. The output features of each convolution block in the lightweight convolution stacking stage are spatially weighted to adjust the denoising intensity in different regions.

[0057] S34. The denoised feature map is upsampled and reconstructed through deconvolution and activation to output a denoised image. The denoised image retains the target frame L. t The core scene information, namely the spray structure information, also incorporates temporal redundancy information;

[0058] In this embodiment, to avoid the spray filament boundaries being overly smoothed during the backbone denoising process of the improved FFDNet denoising network, which would fail to meet the high-frequency detail requirements of subsequent edge detection and spray geometry measurement, a shallow feature bypass is set in the improved FFDNet denoising network.

[0059] Shallow feature bypass is derived from the first or second convolutional block of the convolutional feature extraction layer of the improved FFDNet denoising network, and after basic noise suppression, it bypasses to output a high-frequency feature map H. t Shallow convolutional features retain more edge, texture, and high-frequency information, which is beneficial for extracting thin spray boundaries. In some possible embodiments, the shallow feature bypass includes a 1×1 convolutional dimensionality reduction layer, a lightweight convolutional enhancement layer, and an upsampling layer, the output of which is a high-frequency feature map H with the same size as the denoised image. t ;

[0060] The shallow feature bypass shares some shallow convolution parameters with the backbone network and can be updated together with the backbone network during closed-loop training without additional training; thus, the improved FFDNet denoising network outputs both denoised images and high-frequency feature maps.

[0061] S4. Using the denoised image and high-frequency feature image as input, refine the spray edge;

[0062] First, the detailed feature information of the denoised image and the high-frequency feature image are fused using the Sobel edge detection operator to output an edge prior map. Feature encoding is then performed on the denoised image and the high-frequency feature image to obtain denoised features and high-frequency features. Subsequently, a fused feature map is generated through a gated fusion module. The gate weights are spatially adaptive, generated by convolutional layers and the Sigmoid activation function. Image regions are distinguished based on the edge prior, and fusion weights for the two sets of features are adaptively generated at different spatial locations. The weights for high-frequency features are increased in edge regions, and the weights for denoised features are increased in flat regions. The fused feature map is then processed by convolution and activation to obtain a continuous edge probability map P. t This serves as the input for the subsequent learnable nonmaximum suppression module;

[0063] Secondly, this embodiment refines the spray edge probability map through a learnable non-maximum suppression module, adaptively suppressing non-edge responses in the edge gradient direction. Specifically, the learnable non-maximum suppression module includes a gradient direction estimation unit, a neighborhood response comparison unit, and a learnable threshold unit. The gradient direction estimation unit calculates the spray edge probability map P. t The gradient magnitude and gradient direction are determined; the neighborhood response comparison unit selects neighboring pixels on both sides of the center pixel along the gradient direction and calculates the weighted response value of the neighboring pixels; for a pixel (x,y), if the response value of the center pixel is less than or equal to the weighted response value of the neighboring pixels along the gradient direction, or if the response value of the center pixel is lower than the learnable threshold, then the center pixel is suppressed; otherwise, the center pixel is retained as a candidate boundary pixel.

[0064] In some possible implementations, the learnable nonmaximum suppression module employs a Sigmoid gating function to achieve differentiable boundary screening:

[0065] ;

[0066] in, This is the refined boundary response. Here, T is the neighborhood-weighted response value, T is the learnable threshold, and k is the slope coefficient. For the Sigmoid function;

[0067] The learnable nonmaximum suppression module participates in backpropagation, enabling the boundary refinement process to be co-optimized with the geometric measurement target.

[0068] S5. Fit the spray boundary and measure the geometric parameters based on the spray thin boundary diagram B;

[0069] Taking the nozzle outlet reference point as the spray starting point O(x0,y0), the refined spray thin boundary map B is scanned along the spray diffusion direction. The pixel with the highest response value on the boundary map is extracted as the candidate boundary point. Isolated noise points are removed by the outlier removal method to obtain the left boundary point set and the right boundary point set.

[0070] Let the set of left boundary points be The right boundary point set is Perform line fitting on the left boundary point set and the right boundary point set respectively:

[0071] ;

[0072] ;

[0073] in, and The slope and intercept of the line fitted to the left boundary. and The slope and intercept of the line fitted to the right boundary;

[0074] spray cone angle ;

[0075] The spray penetration distance D is calculated based on the distance from the spray initiation point to the farthest boundary point of the spray:

[0076] ;

[0077] in s is the image scale;

[0078] In one implementation, the boundary fitting process uses a differentiable singular value decomposition fitting method, so that the spray cone angle and spray penetration distance can be used as differentiable outputs to participate in closed-loop training.

[0079] Specifically, the optimal fitting line parameters corresponding to the boundary point set are calculated using singular value decomposition, and the spray cone angle and spray penetration distance are further calculated based on the fitting results of the left and right boundaries. Since the boundary points originate from the edge probability map output by the network, and the boundary fitting process, the spray geometric parameter calculation process, and the geometric loss calculation process all participate in the network training, the spray geometric parameter error can be backpropagated to the edge probability map generation module, the boundary refinement module, and the improved FFDNet denoising network during the backpropagation process, thereby achieving end-to-end joint optimization based on the spray geometric measurement target. As a result, the network training target is further transformed from traditional image visual quality optimization to spray geometric measurement accuracy optimization, improving the stability and accuracy of the spray cone angle and spray penetration distance measurement results.

[0080] S6. Construct a multi-task loss function and perform backpropagation optimization;

[0081] This embodiment constructs a differentiable closed-loop training process with the primary objective of optimizing spray geometry measurement, and designs a multi-task loss function: Where L is the total loss. , , and These are the weights of each loss; L g To account for geometric parameter loss, the measurement error is calculated by comparing the predicted and labeled values ​​of the spray geometric parameters. The measurement error is a weighted combination of the spray cone angle error and the penetration distance error; L e For edge detection loss, a weighted combination of cross-entropy loss and Dic loss can be used to constrain the spray edge probability map P. t Consistency with edge annotations; L d For image denoising, L1 loss, L2 loss, or structural similarity loss can be used to constrain the consistency between the denoised image and the reference image; L t This is a temporal consistency loss used to constrain the smoothness of the spray cone angle curve and spray penetration distance curve between consecutive target frames;

[0082] In this embodiment, the geometric parameter loss L gBy assigning the highest weights, the network training objective shifts from optimizing traditional image quality to optimizing spray geometry measurement. During training, three spray images are input, and the spray cone angle and spray penetration distance are obtained through lightweight optical flow pyramid alignment, spatially variable noise map generation, denoising with an improved FFDNet denoising network, shallow high-frequency bypass output, boundary probability map generation, learnable non-maximum suppression, boundary fitting, and geometric parameter calculation. The loss is calculated based on the labeled values ​​of the spray geometry parameters, and the trainable parameters of the improved FFDNet denoising backbone network, shallow feature bypass, gated fusion module, and learnable non-maximum suppression module are iteratively optimized through backpropagation to avoid over-denoising and improve the accuracy of thin edge recognition. Thus, through the "measurement differentiability closed-loop" mechanism of backpropagation, the training and optimization objective of the FFDNet network is changed from PSNR optimization to geometric measurement optimization, while taking into account the systematic error and temporal jitter error of the spray cone angle and penetration distance. Through multiple rounds of closed-loop optimization training, the geometric parameter measurement results are more stable and accurate.

[0083] Furthermore, the software product of this image recognition and measurement method is stored in a storage medium, including several instructions to cause a computer device, such as, but not limited to, a personal computer, a server, or a network device, to store a computer program. When the computer program is executed by a processor, it can execute the FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation described in any embodiment of this application.

[0084] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for spray image recognition and measurement based on three-frame aligned motion compensation using FFDNet, characterized in that, Includes the following steps and content: S1, acquire three images adjacent to the spray image, including a target frame L t , a front frame L t-1 , and a rear frame L t+1 ; S2, estimate first displacement field of the front frame L t-1 to the target frame L t and second displacement field of the back frame L t+1 to the target frame L t , and perform motion compensation on the front frame L t-1 and the back frame L t+1 according to the first displacement field and the second displacement field respectively to generate first aligned frame L1 and second aligned frame L2 corresponding to the target frame L t . S3、according to the target frame L t generate a spatially variable noise map, and input the target frame L t , the first aligned frame L1, the second aligned frame L2 and the spatially variable noise map as multi-channel input features into an improved FFDNet denoising network, wherein the improved FFDNet denoising network outputs a denoised image as a main output, and outputs a high-frequency feature map as a bypass output of a shallow layer feature. S4. The denoising features of the denoised image and the high-frequency features of the high-frequency feature map are fused to generate a spray edge probability map. The spray edge probability map is then refined into a thin edge map by a learnable non-maximum suppression module. S5. Based on the nozzle structure parameter constraints, scan and extract the left and right spray boundary point sets of the spray thin boundary map along the spray diffusion direction, fit the boundary straight line, and measure the spray geometric parameters.

2. The FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation according to claim 1, characterized in that: The spatially variable noise map is a noise intensity distribution map generated based on a differential weighting strategy, in which low noise weights are assigned to pixel groups with high gradients at the edges, and high noise weights are assigned to pixel groups in high variance or overexposed areas.

3. The FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation according to claim 1, characterized in that: Methods for generating the first alignment frame L1 and the second alignment frame L2: S21, down-sampling the target frame L t , the previous frame L t-1 , and the next frame L t+1 to a first scale, calculating a coarse displacement field at the first scale to obtain an initial optical flow; S22. Upsample the initial optical flow to the second scale, and construct a local correlation window near the aligned pixel pairs, and calculate the residual displacement based on the local correlation window; S23. Superimpose the initial optical flow and the residual displacement to obtain the first displacement field and the second displacement field; S24. Based on the first displacement field and the second displacement field, the previous frame L is resampled using a reverse resampling method. t-1 and the following frame L t+1 Mapped to the target frame L respectively t Using a coordinate system, the first alignment frame L1 and the second alignment frame L2 are obtained.

4. The FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation according to claim 3, characterized in that: The target frame L t The first aligned frame L1, the second aligned frame L2, and the spatially variable noise map are spliced ​​together in the channel dimension to form a four-channel input feature, which is then input into the improved FFDNet denoising network. The improved FFDNet denoising network includes a downsampling layer, a convolutional feature extraction layer, a spatial modulation layer, and an upsampling reconstruction layer. The noise reduction methods include: S31. The downsampling layer performs feature downsampling on the four-channel input features; S32, The convolutional feature extraction layer extracts spray structure features, noise features, and temporal redundancy features; S33. The spatial modulation layer concatenates the noise feature map and the temporal feature map, and generates spatial modulation coefficients to weight and adjust the feature responses of different spatial regions; wherein, the noise feature map is generated based on the convolutional encoding of the spatially variable noise map, and the temporal feature map is generated by calculating the first aligned frame L1 and the target frame L1 respectively. t and the second aligned frame L2 and the target frame L t Output of difference information; S34. Upsample and reconstruct the denoised feature map after denoising, and output the denoised image that incorporates temporal redundancy information.

5. The FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation according to claim 4, characterized in that: The shallow feature bypass is derived from the shallow convolutional block of the convolutional feature extraction layer, and after convolutional dimensionality reduction and upsampling processing, the high-frequency feature map with the same size as the denoised image is output.

6. The FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation according to claim 5, characterized in that: The method for generating the spray edge probability map is as follows: the denoised image and the high-frequency feature map are respectively feature-encoded, and a fused feature map is generated through a gated fusion mechanism. The gated fusion mechanism is used to increase the fusion weight of the high-frequency feature map in the spray edge region and increase the fusion weight of the denoised image in the background region; the spray edge probability map is output according to the fused feature map.

7. The FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation according to claim 6, characterized in that: The learnable nonmaximum suppression module includes a gradient direction estimation unit, a neighborhood response comparison unit, and a learnable threshold unit; the gradient direction estimation unit is used to calculate the gradient direction of the spray edge probability map; the neighborhood response comparison unit is used to compare the response values ​​of neighboring pixels along the gradient direction; The learnable threshold unit is used to filter candidate boundary pixels based on a learnable threshold in order to output the spray thin boundary map.

8. The FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation according to claim 1, characterized in that: The boundary line fitting process in step S5 adopts a differentiable singular value decomposition fitting method.

9. The FFDNet spray image recognition and measurement method based on three-frame aligned motion compensation according to claim 8, characterized in that: It also includes step S6, constructing a multi-task loss function with the primary objective of optimizing spray geometry measurement and performing backpropagation optimization: The multi-task loss function: ; Where L is the total loss; , , and These are the weights of each loss, and L g For geometric parameter loss; L e For edge detection loss; L d Image denoising loss; L t The timing consistency loss is used to constrain the consecutive target frames L. t The smoothness of the curves showing the changes in spray cone angle and spray penetration distance.

Images